Long tail image recognition method based on pseudo label

Abstract

The application discloses a long-tail image recognition method based on pseudo labels, which comprises the following steps: two different data enhancement methods are used on input pictures to obtain two groups of samples, which are respectively input into an encoder and a momentum encoder to obtain two groups of feature representations; the first group of feature representations is input into a classifier, and a weighted cross-entropy loss is calculated according to the real labels of the images; the two groups of feature representations are respectively subjected to different nonlinear mapping and two-norm normalization; the second group of feature representations and the prediction confidence of the classifier are stored in the feature cache area corresponding to the category; a weighted cosine similarity loss is calculated; the encoder and the classifier are optimized by the stochastic gradient descent, and the momentum encoder is optimized by the momentum. In the natural species identification task, the pseudo labels generated by the classifier are used to augment the sample amount of rare species, and a more comprehensive feature representation is learned by using self-supervised training, so that the image recognition accuracy of the species in the natural scene can be improved.

Description

Technical Field

[0001] This invention belongs to the field of class-imbalanced image recognition, specifically relating to a long-tail image recognition method based on pseudo-labels in natural scenes. Background Technology

[0002] Imbalanced data distribution is prevalent in nature. In image classification research, imbalanced data, especially data with a long-tailed distribution of sample size, is currently a popular research area, and its research content and results meet the needs of practical applications. Imbalanced image data refers to datasets where different categories occupy unequal numbers of images. In long-tailed data distributions, a very small number of categories (head categories) occupy the majority of the samples in the dataset, while most categories (tail categories) occupy only a small number of images.

[0003] Neural network models trained on imbalanced datasets often perform poorly on tail class data, primarily due to the limited number of images available for this category. During training, the majority of the training data is allocated to head class images, leaving far fewer tail class images for the model. To address this issue, researchers employ resampling strategies. These strategies involve oversampling tail class samples or undersampling head class samples during training to increase the amount of tail class data utilized by the network, thereby enhancing its ability to identify tail classes.

[0004] While resampling techniques have achieved good classification results on imbalanced data, increasing the sampling frequency of tail class data with the total amount of data remains unchanged means reducing the sampling frequency of head class data. This prevents the model from fully utilizing the head class data, which contains more information, and undermines the model's representation learning. Summary of the Invention

[0005] The purpose of this invention is to provide a long-tail image recognition method based on pseudo-labels in natural scenes.

[0006] The technical solution to achieve the objective of this invention is as follows: Firstly, this invention provides a method for training long-tailed image data based on pseudo-labels, comprising the following steps:

[0007] Step 1: Obtain species image data samples from natural scenes. Input the two sets of samples, which have undergone two different data augmentations, into the encoder and momentum encoder respectively to obtain feature representations. and feature representation ;

[0008] Step 2, Represent the features Input the classifier and calculate the weighted cross-entropy loss;

[0009] Step 3, represent the features respectively and Input different nonlinear mappers and Then, L2 norm normalization is performed to obtain a new feature representation. and ,calculate and The cosine similarity loss between them is used, and the prediction result of the classifier is used as the result. The pseudo-tags, and based on the pseudo-tags, and prediction confidence Store them together in the feature cache area;

[0010] Step 4, based on the sample's true label Randomly select a feature representation from the cache. ,calculate and Weighted cosine similarity loss between them;

[0011] Step 5: Update the encoder, classifier, and nonlinear mapper using stochastic gradient descent. The parameters are updated simultaneously using the momentum update method to update the momentum encoder and the nonlinear mapper. The parameters are used to complete the training and save the encoder and classifier parameters.

[0012] In a second aspect, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method described in the first aspect.

[0013] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0014] Fourthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0015] Compared with the prior art, the significant advantages of this invention are: (1) it transfers some head class data to tail class in the form of pseudo-labels, which can increase the proportion of tail class data and make full use of the rich semantic information in head class data; (2) it discovers more comprehensive image features through self-supervised learning based on cosine similarity loss, so that the model has stronger generalization ability. Attached Figure Description

[0016] Figure 1 This is a flowchart of the long-tail image recognition method based on pseudo-labels of the present invention. Detailed Implementation

[0017] This invention proposes a long-tail image recognition method based on pseudo-labels, comprising the following steps: Two different data augmentation methods are applied to the input image to obtain two sets of samples; the two sets of samples are input into an encoder and a momentum encoder, respectively, to obtain two sets of feature representations; the first set of feature representations is input into a classifier for classification, and a weighted cross-entropy loss is calculated based on the image's true label, with the weights calculated according to the proportion of the image's category in the dataset; the two sets of feature representations are subjected to different nonlinear mappings and L2 normalization; the second set of feature representations and the classifier's prediction confidence are stored in the corresponding category's feature cache based on the pseudo-labels predicted by the classifier; a cosine similarity loss is calculated for the two sets of feature representations; a feature representation is randomly selected from the corresponding cache based on the image's true category, and a weighted cosine similarity loss is calculated between it and the first set of feature representations, with the weight being the classification confidence stored in the cache; stochastic gradient descent optimization is performed on the encoder and classifier based on the above losses, and momentum optimization is performed on the momentum encoder. In natural species identification tasks, this invention utilizes pseudo-labels generated by a classifier to augment the sample size of rare species and employs self-supervised training to learn more comprehensive feature representations, thereby improving the accuracy of species image recognition in natural scenes.

[0018] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings.

[0019] Combination Figure 1 A long-tail image recognition method based on pseudo-labels, specifically including the following steps:

[0020] Step 1: Use a terminal device to collect a dataset of species images in natural scenes;

[0021] Step 2: Input the two sets of samples, which have undergone two different data augmentations, into the encoder and momentum encoder respectively to obtain feature representations. and feature representation ;

[0022] The data augmentation used includes AutoAugment, random horizontal flipping, random adjustments to image brightness, contrast, saturation, grayscale, and random Gaussian blur. AutoAugment randomly selects one of the following for image enhancement each time: histogram equalization, inversion, skew, rotation, sharpening, brightness adjustment, and color adjustment. The encoder and momentum encoder are convolutional neural networks with identical structures and initial parameters.

[0023] Step 3, Represent the features Input the classifier and calculate the weighted cross-entropy loss;

[0024] Where the weight is ,in To input the image category, For categories in the training set The sample size is used to calculate the cross-entropy loss by adding weights to the classifier output.

[0025] Step 4, represent the features respectively. and Input different nonlinear mappers and Then, L2 norm normalization is performed to obtain a new feature representation. and ,calculate and The cosine similarity loss between them is used, and the prediction result of the classifier is used as the result. The pseudo-tags, and based on the pseudo-tags, and prediction confidence Store them together in the feature cache area;

[0026] The nonlinear mapper consists of a linear classifier, batch normalization, and ReLU activation. and The structure and initialization parameters are the same. The feature buffer maintains a fixed-capacity feature queue for each category; each time a new feature is added, the oldest feature added to the queue is deleted. The prediction confidence is obtained by applying sigmoid activation to the classifier output obtained in step 2. The cosine similarity loss is...

[0027]

[0028] in Hyperparameters used to control the loss weights.

[0029] Step 5, based on the sample's true label Randomly select a feature representation from the cache. ,calculate and Weighted cosine similarity loss between them;

[0030] Indicates pseudo-tags as The feature representation. The weights in the weighted cosine similarity loss are the prediction confidence obtained in step 4, and the weighted cosine similarity loss is...

[0031]

[0032] in Hyperparameters used to control the loss weights.

[0033] Step 6: Update the encoder, classifier, and nonlinear mapper using stochastic gradient descent. The parameters are updated simultaneously using the momentum update method to update the momentum encoder and the nonlinear mapper. The parameters are used to complete the training and save the parameters of the encoder and classifier;

[0034] Momentum update method is

[0035]

[0036] in These are the parameters of the momentum encoder. For encoder parameters, The hyperparameters used to control the momentum update rate.

[0037] Step 7: Load model parameters to identify natural species images.

[0038] As another implementation method, the structure of the encoder and momentum encoder can be designed according to actual needs.

[0039] As an alternative implementation, stochastic gradient descent can be replaced by other parameter optimization methods.

[0040] As an alternative implementation, the natural species image dataset can be replaced with long-tail image data from other fields, depending on the specific application requirements.

Claims

1. A long-tail image recognition method based on pseudo-labels, characterized in that, Includes the following steps: Step 1: Obtain species image data samples in natural scenes. Input the two sets of samples, which have undergone two different data augmentations, into the encoder and momentum encoder respectively to obtain feature representation s and feature representation s'. Step 2: Input the feature representation s into the classifier and calculate the weighted cross-entropy loss; the weights of the cross-entropy loss are ln(N). c / ∑ j N j ), where c is the input image category, N c To calculate the cross-entropy loss, we add weights to the classifier output to determine the sample size of class c in the training set. Step 3: Input the feature representations s and s' into different nonlinear mappers p1 and p2 respectively, and perform L2 normalization to obtain new feature representations f. x and f x' Calculate f x and f x' The cosine similarity loss between them is used, and the prediction result of the classifier is used as f. x' The pseudo-tags, and based on the pseudo-tags, f x' and prediction confidence w x Store them together in the feature cache area; Step 4: Randomly select a feature representation f from the cache based on the true label c of the sample. c' Calculate f x with f c' The weighted cosine similarity loss between them; the weights in the weighted cosine similarity loss are the prediction confidence obtained in step 3, and the weighted cosine similarity loss is... Where β is a hyperparameter that controls the loss weights; Step 5: Update the parameters of the encoder, classifier, and nonlinear mapper p1 using stochastic gradient descent, and simultaneously update the parameters of the momentum encoder and nonlinear mapper p2 using momentum update. Complete the training and save the encoder and classifier parameters; the momentum update method is θ. k ←γθ k +(1-γ)θ q Where θ k For the parameters of the momentum encoder, θ q Here, γ represents the encoder parameters, and γ is a hyperparameter controlling the momentum update rate.

2. The long-tail image recognition method based on pseudo-labels according to claim 1, characterized in that, The data augmentation used in step 1 includes AutoAugment, random horizontal flipping, random changes to image brightness, contrast, saturation, grayscale, and random Gaussian blur; AutoAugment randomly selects one of histogram equalization, inversion, tilting, rotation, sharpening, brightness adjustment, and color adjustment to enhance the image each time; the encoder and momentum encoder are convolutional neural networks with the same structure and initial parameters.

3. The long-tail image recognition method based on pseudo-labels according to claim 1, characterized in that, In step 3, the nonlinear mapper consists of a linear classifier, batch normalization, and ReLU activation; the structures of p1 and p2 are the same as the initialization parameters; the feature buffer maintains a fixed-capacity feature queue for each category, and each time a new feature is stored, the earliest feature added to the queue is deleted.

4. The long-tail image recognition method based on pseudo-labels according to claim 3, characterized in that, The prediction confidence is obtained by applying sigmoid activation to the classifier output obtained in step 2. Cosine similarity loss is Where α is a hyperparameter that controls the loss weights.

5. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the pseudo-label-based long-tail image recognition method according to any one of claims 1 to 4.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the pseudo-label-based long-tail image recognition method according to any one of claims 1 to 4.

7. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method described in any one of claims 1-4.

Images