A long-tail distribution image classification method based on multi-objective optimization

Abstract

This invention discloses a long-tailed distribution image classification method based on multi-objective optimization. Step S1: Obtain the predicted values ​​Z of samples {x,y} of each class k in a batch. k Step S2: Calculate the loss value for each category in the current batch; Step S3: Convert the loss value L of each category of samples in the current batch... k Backpropagation is performed one by one to solve for the network gradient G corresponding to each type of sample. k Step S4: Input the normalized network gradient into the FRANKWOLFESOLVER module to calculate the weights α corresponding to each type of sample in the current batch. k Step S5: Use weight α k For each category of loss value L k We obtain the network gradient G by weighted summation. f Using G f Backpropagation is performed to update the network parameters; step S6, repeating steps one to five until training is complete to obtain the optimal model. This optimal model is then used to predict the image category, and the predicted label L is output. Compared with existing technologies, this invention establishes a model through a multi-objective optimization method, ultimately improving the ability to recognize tail categories.

Description

Technical Field

[0001] This invention relates to the field of computer network technology, and specifically to a long-tailed distribution image classification method based on multi-objective optimization. Background Technology

[0002] With advancements in hardware and software, a wealth of research has emerged in the field of artificial intelligence, particularly deep learning. Deep learning has also been extensively applied in computer vision. Image classification is one of the classic tasks in computer vision, aiming to assign category labels to input images within a predefined label space using a network model. Since the maturation of deep learning technology, image classification has received widespread attention and research.

[0003] Most current image classification methods use pre-trained networks such as ResNet, ResNetXt, and VGG as the backbone of the model to extract image features, and then design a classifier accordingly to complete the classification task. These image classification methods are mostly based on balanced class datasets such as ImageNet, CIFAR10, and CIFAR100, where the sample size for each class is equal. However, in the real world, data distribution follows a long-tailed distribution, which is class-imbalanced, meaning some classes have a large number of samples while others have very few. This class imbalance causes deep convolutional neural network models to bias towards learning the classes with large sample sizes during training, while learning the classes with smaller sample sizes is relatively insufficient. This leads to poor recognition performance of the models on classes with fewer training samples during testing. Therefore, in recent years, a large amount of research on deep long-tailed learning has been conducted for image classification tasks with long-tailed distributions.

[0004] Current research on deep long-tail learning problems focuses on class rebalancing, including methods such as resampling and cost-sensitive learning. Resampling methods adjust training by controlling the sampling probability of each class, typically increasing the sampling probability of tail classes. However, this approach can lead to overfitting to tail classes. Cost-sensitive learning achieves class rebalancing by adjusting the loss values ​​of different classes during training; however, while improving the model's ability to recognize tail classes, it can impair its ability to recognize head classes. Summary of the Invention

[0005] The purpose of this invention is to propose a long-tailed image classification method based on multi-objective optimization. By using the idea of ​​multi-objective optimization, the loss function of each category is used as the optimization objective. During the model training process, the loss function of all categories can be reduced, which solves the problem that the gradient of the tail category is suppressed by the gradient of the head category during training. This better balances the gradient contributions of the head category and the tail category, thereby improving the recognition effect of the tail category while improving the overall recognition effect of the model.

[0006] This invention is achieved using the following technical solution:

[0007] A long-tailed distribution image classification method based on multi-objective optimization, the method includes the following steps:

[0008] Step S1, Training Set Randomly sample a batch of data without repetition. Get all categories in a batch samples Predicted value The dataset in this batch is then fed into a pre-trained network of a model with ResNet32 as the backbone.

[0009] Step S2: Calculate the loss values ​​for each category in the current batch, specifically including the following steps:

[0010] Step S2.1: Predicted values ​​for each category in the current batch. The Softmax function is used to process each value to obtain the corresponding predicted probability. , ;

[0011] Step S2.2: Select each category included in the current batch. Predicted probability value and its corresponding label vector The input is fed into the cross-entropy function to calculate the loss value of each class of samples in the current batch. , , Represents the set of real numbers;

[0012] Step S3: Set the loss values ​​for each type of sample in the current batch. Backpropagation is performed one by one to solve for the network gradient corresponding to each type of sample. ;

[0013] Step S4: Use L2 Normalization to normalize the current batch of each category Corresponding network gradient Normalization Normalized network gradient The input is fed into the FRANKWOLFESOLVER module to calculate the weights of each type of sample in the current batch. ;

[0014] Step S5: Use the calculated weights Loss values ​​for each category Weighted summation yields the final network gradient. ,use Backpropagation and updating of network parameters;

[0015] Network gradient The formula is as follows:

[0016]

[0017] in, For category The set representation;

[0018] Step S6: Repeat steps one through five until training is complete, obtain the optimal model, use the optimal model to predict the image category, and output the label of the prediction result. .

[0019] Compared with existing technologies, the long-tailed distribution image classification method based on multi-objective optimization of the present invention can achieve the following beneficial effects:

[0020] (1) This invention is based on the theory of multi-objective optimization and has theoretical protection;

[0021] (2) In the model training process, the present invention uses a multi-objective optimization method to ensure that the direction of each gradient descent does not damage any objective function, thereby avoiding the gradient of the tail category being submerged by the gradient of the head category, so that the loss function of the tail category can also decrease, and ultimately improve the recognition ability of the tail category. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the overall process of a long-tailed distribution image classification method based on multi-objective optimization according to the present invention;

[0023] Figure 2 This is a data flow diagram of a long-tailed distribution image classification method based on multi-objective optimization according to the present invention. Detailed Implementation

[0024] The technical solutions of the present invention will now be clearly and completely described with reference to the accompanying drawings. All other embodiments of the technical solutions obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0025] like Figure 1 The diagram shown is a schematic representation of the overall process of a long-tailed distribution image classification method based on multi-objective optimization according to the present invention.

[0026] Step S1: Obtain the predicted values ​​for each category of samples in a batch, specifically including the following steps:

[0027] Step S1.1: From the training set Randomly sample a batch of data without repetition. , , , This indicates the first data point in the batch obtained from sampling. Image data of one sample image, Indicates the first The sample data for each sample image is a 3-dimensional... H W is a real matrix. This indicates the height of the image sample. This represents the width of the image, and 3 represents the three channels of the image: Red, Green, and Blue. Represents the set of real numbers. This indicates the first data point in the sampled batch. Labels for each sample image. This represents the total number of categories in the dataset. This indicates the number of samples contained in a batch. This indicates the sample image number, and the dataset for this batch is input into the model pre-training network with ResNet32 as the backbone;

[0028] Step S1.2: Obtain the prediction model ,in, This represents the input image of the model. This represents the parameter set of the neural network model and the predicted values ​​for each type of sample in the current batch. , ,in, This represents all categories in the current batch of data. The number of samples, , .

[0029] Step S2: Calculate the loss values ​​for each category in the current batch, specifically including the following steps:

[0030] Step S2.1: Predicted values ​​for each category in the current batch. The Softmax function is used to process each value to obtain the corresponding predicted probability. , ;

[0031] Step S2.2: Include all categories in the current batch. Predicted probability value and its corresponding label vector ( The label vector expression is input into the cross-entropy function to calculate the loss value of each class of samples in the current batch. , ;

[0032] Note 1: Predicted probability values ​​for each type of sample in the current batch. The solution process is as follows:

[0033] Assume the dataset has a total of Categories Indicates the current batch All categories in a sample A sample is input into the model to obtain the output. The softmax function is used to calculate the predicted probability value of the sample after it is input into the model. The formula is as follows:

[0034]

[0035] in, Indicates a category The corresponding predicted value, .

[0036] Note 2: Calculate the loss value for each type of sample in the current batch. The process is as follows:

[0037] Calculate a certain sample { x,y loss value The formula is as follows:

[0038]

[0039] Calculate the total in the current batch Categories Loss value per sample The formula is as follows:

[0040]

[0041] in, Category in the current batch The number of samples, For the first i The loss value for each sample;

[0042] Step S3: Set the loss values ​​for each type of sample in the current batch. Backpropagation is performed one by one to solve for the network gradient corresponding to each type of sample. ;

[0043] Step S4: Use L2 Normalization to normalize the current batch of each category Corresponding network gradient Normalization Normalized network gradient The input is fed into the FRANKWOLFESOLVER module to calculate the weights of each type of sample in the current batch. ;

[0044] Note 3: The normalization process for gradients of each type of network is as follows:

[0045] Assumption Normalize each parameter to obtain the category. Corresponding normalized network gradient The formula is as follows:

[0046]

[0047]

[0048] in, In the neural network model, the first... The gradient of a learnable parameter, This represents the total number of parameters in the neural network model.

[0049] Step S5: Use the calculated weights Loss values ​​for each category Weighted summation yields the final network gradient. The formula is as follows:

[0050]

[0051] use Backpropagation and updating of network parameters;

[0052] Step S6: Repeat steps one through five until training is complete, obtain the optimal model, use the optimal model to predict the image category, and output the label of the prediction result. The training process for this step is illustrated with an example below:

[0053] For one training round, repeat steps one through five until all data in the training set is collected; (2) Repeat training for 200 rounds, using the validation set at the end of each round. Test the current model, and save the model with the best test results as the final model; during prediction, first load the model with the best test results, and then input the image to be predicted. The model outputs the final predicted label. .

[0054] The table shows a comparison of experimental results between the present invention and traditional image classification methods on CIFAR10-LT with three imbalance ratios. This comparison demonstrates that the multi-objective optimization-based model of the present invention achieves greater performance improvement on CIFAR10-LT.

[0055] Table 1

[0056]

[0057] The feasibility of the method of the present invention is verified below with specific examples, as detailed in the following description:

[0058] The method of this invention was validated on the CIFAR10-LT dataset. The original CIFAR10 consists of 10 classes and contains 60,000 images of size 32. 32 color images. 50,000 images were used for training, and the remaining images were used for validation. CIFAR10-LT utilizes an exponential function. , , where, It is a category index. It is a category The number of samples in the original CIFAR training set and The validation set remains unchanged. Imbalance ratio Defined as the ratio of the sample size of the most frequent class to the sample size of the least frequent class, i.e. , In long-tail image classification Often set to common values, i.e. .

[0059] The experiment used the top-1 accuracy of the prediction results to quantitatively evaluate the classification results; the higher the top-1 accuracy value, the better.

Claims

1. A long-tailed distribution image classification method based on multi-objective optimization, characterized in that, The method includes the following steps: Step S1, Training Set Randomly sample a batch of data without repetition. Get all categories in a batch samples Predicted value The dataset in this batch is then fed into a pre-trained network of a model with ResNet32 as the backbone. Step S2: Calculate the loss values ​​for each category in the current batch, specifically including the following steps: Step S2.1: Predicted values ​​for each category in the current batch. The Softmax function is used to process each value to obtain the corresponding predicted probability. , ; Step S2.2: Select each category included in the current batch. Predicted probability value and its corresponding label vector The input is fed into the cross-entropy function to calculate the loss value of each class of samples in the current batch. , , Represents the set of real numbers; Step S3: Set the loss values ​​for each type of sample in the current batch. Backpropagation is performed one by one to solve for the network gradient corresponding to each type of sample. ; Step S4: Use L2 Normalization to normalize the current batch of each category Corresponding network gradient Normalization Normalized network gradient The input is fed into the FRANKWOLFESOLVER module to calculate the weights of each type of sample in the current batch. ; Step S5: Use the calculated weights Loss values ​​for each category Weighted summation yields the final network gradient. ,use Backpropagation and updating of network parameters; Network gradient The formula is as follows: in, For category The set representation; Step S6: Repeat steps one to five until training is complete, obtain the optimal model, use the optimal model to predict the image category, and output the label of the prediction result. .

2. The long-tailed distribution image classification method based on multi-objective optimization as described in claim 1, characterized in that, Solve for the predicted probability values ​​of each type of sample in the current batch. The process is as follows: The dataset has a total of Categories This is the current batch. Class of samples A sample is input into the model to obtain the output. The softmax function is used to calculate the predicted probability value obtained after inputting the sample into the model. The formula is as follows: in, Indicates a category The corresponding predicted value.

3. The long-tailed distribution image classification method based on multi-objective optimization as described in claim 1, characterized in that, Solve for the loss values ​​of each type of sample in the current batch. The process is as follows: Calculate a certain sample { x,y loss value The formula is as follows: Calculate the total in the current batch Categories Loss value per sample The formula is as follows: in, Category in the current batch The number of samples, For the first i The loss value for each sample.

4. The long-tailed distribution image classification method based on multi-objective optimization as described in claim 1, characterized in that, The normalization process for gradients of each type of network is as follows: By analyzing each category Network gradient corresponding to the sample Normalization is performed to obtain the corresponding normalized network gradient. The formula is as follows: in, In the neural network model, the first... The gradient of a learnable parameter, This represents the total number of parameters in the neural network model.

5. The long-tailed distribution image classification method based on multi-objective optimization as described in claim 1, characterized in that, In step 1 Indicates the first The sample data for each sample image is a 3-dimensional... H W is a real matrix. This indicates the height of the image sample. This indicates the width of the image, and 3 represents the R, G, and B channels of the image. Represents the set of real numbers. This indicates the first data point in the batch obtained from sampling. Labels for each sample image. This represents the total number of categories in the dataset. This indicates the number of samples contained in a batch.

Images