Adaptive weighted long-tail classification method and device based on effective area of ​​feature space

Abstract

The present invention discloses an adaptive weighted long-tail classification method and device based on the effective area of ​​feature space, the method comprising: training a feature extractor and a classifier on the distribution of original data to obtain a basic classification model; based on the trained feature extractor, extracting the feature representation of the original data, and constructing a correlation coefficient matrix of the category according to the correlation coefficient between the feature representations in each category; based on the correlation coefficient matrix, calculating the effective area of ​​each category in the feature space, and assigning weights to each category according to the effective area, training the basic classification model to obtain a classification model; based on the classification model, obtaining the classification result of the data to be detected. The present invention estimates the effective area in the feature space to give different weight losses to different categories, better divides the feature space, and classifies the long-tail data more effectively.

Description

Technical Field

[0001] The present invention relates to the field of image data classification, and in particular to an adaptive weighted long-tail classification method and device based on effective area of ​​feature space. Background Art

[0002] With the rapid development of data acquisition technology, deep neural networks can achieve excellent performance on large-scale evenly distributed training data. However, in real-world scenarios, data is usually distributed in a long-tailed manner, that is, most categories belong to the tail categories with only a small number of samples, while a few head categories have most samples. In recent years, image classification for long-tailed data has been an important task in the image field, and effectively using long-tail data to train good classifiers has received increasing attention.

[0003] Existing methods generally use rebalancing strategies and reweighting methods to improve the classification accuracy of tail categories.

[0004] The rebalancing strategy achieves a balance in the number of training samples through resampling or data augmentation methods, including resampling and data augmentation methods. The resampling method obtains training data with uniform data distribution by oversampling the tail categories or undersampling the head categories. However, oversampling may cause overfitting by repeating samples in the tail categories, while undersampling may damage feature representation by discarding important head data. In order to alleviate the overfitting problem, data augmentation methods are introduced to enhance the intra-class diversity of the tail categories by generating new samples. However, as the number of enhanced samples increases, the training speed slows down sharply, especially for large-scale data sets. Even though the latest methods can reduce consumption through implicit data augmentation, the rationality of the newly generated enhanced samples remains to be evaluated. Therefore, although the rebalancing strategy alleviates the phenomenon that the classification hyperplane is severely tilted toward the tail categories to a certain extent, due to the small number of samples in the tail categories and the low intra-class diversity, it is easy to cause overfitting or increase the difficulty of training.

[0005] Compared with rebalancing, reweighting is simpler and lighter. It affects the loss function by assigning higher weights to the tail classes, thereby improving the accuracy of the tail classes. The reweighting method gives a greater penalty to the tail samples when calculating the loss function to increase the importance of the tail samples while keeping the original data distribution unchanged. It includes sample-level and category-level weighting methods. The sample-level weighting operation assigns different weights to different samples. Most of the reweighting methods on categories design weights or margins based on the number of samples. However, the goal of fair training is to make the hyperplane evenly divide the feature space, and the number of samples cannot represent the size of the feature space. Therefore, how to better divide the feature space and improve the performance of the model while ensuring both effectiveness and efficiency remains an important issue.

[0006] In summary, facing extremely unbalanced long-tail data, it is a huge challenge to simply and effectively split the feature space to improve classification performance. Summary of the invention

[0007] The purpose of the present invention is to design an adaptive weighted long-tail classification method and device based on the effective area of ​​feature space, by fully analyzing the correlation between samples, estimating the effective area of ​​each category from a new perspective, so that the feature space can be better divided according to the weighted operation of the effective area. The method is divided into two stages. The first stage is to use cross entropy loss to train basic feature extractors and classifiers on the original data distribution, and the second stage uses the model feature extractor of the first stage to extract the features of the training data. Then, the correlation between any two samples in the same category is calculated to form a correlation coefficient matrix R for each category. Then, the present invention uses the formula 1 / a T Ra calculates the effective area for each category, where a i =1 / n, where n is the number of samples corresponding to each category. Finally, the inverse of the effective area is used as the weight to recalculate the cross entropy loss. It is worth noting that since the features are continuously optimized during the training process, the weights defined in the effective area can be updated adaptively in the present invention.

[0008] To achieve the above object, the present invention adopts the following technical solutions:

[0009] An adaptive weighted long-tail classification method based on effective area of ​​feature space, the steps of which include:

[0010] Train the feature extractor and classifier on the distribution of the original data to obtain a basic classification model;

[0011] Extracting feature representations of the original data based on the trained feature extractor, and constructing a correlation coefficient matrix of each category according to correlation coefficients between the feature representations in the category;

[0012] Based on the correlation coefficient matrix, the effective area of ​​each category in the feature space is calculated, and weights are assigned to each category according to the effective area, and the basic classification model is trained to obtain a classification model;

[0013] Based on the classification model, a classification result of the data to be detected is obtained.

[0014] Furthermore, the method of training the feature extractor and the classifier on the distribution of the original data includes: training based on the cross entropy loss.

[0015] Furthermore, constructing a correlation coefficient matrix of each category according to the correlation coefficients between the feature representations in the category includes:

[0016] For category l c , calculate the covariance of any two feature representations;

[0017] Calculation Category c The variance of

[0018] Based on the covariance and category l c The variance of , obtains the correlation coefficient between any two feature representations;

[0019] Based on the correlation coefficients, the correlation coefficient matrix is ​​constructed.

[0020] or,

[0021] Calculation Category c The class center μ c ;

[0022] By taking the centralized {f i -μ c} and {f j -μ c} as each item of the correlation coefficient matrix to obtain the correlation coefficient matrix of this category, where f i 、f j Divided into original data x i 、x j The feature representation of .

[0023] Furthermore, the calculating the effective area of ​​each category in the feature space based on the correlation coefficient matrix includes:

[0024] Get vector a c , where the vector a c The i-th element of N c For category l c The number of features represented by the

[0025] According to the vector a c With the correlation coefficient matrix, calculate the category l c The effective area in feature space.

[0026] Furthermore, the weight Where |C| represents the number of categories, Indicates category l c The effective area in feature space.

[0027] Furthermore, the loss function of the basic classification model is trained N represents the number of raw data, Represents the original data x i In category l cThe prediction score on .

[0028] Furthermore, when training the basic classification model, the effective area of ​​each category is obtained based on the sum of the effective areas of all batches in the feature space, the weight of each category is calculated based on the inverse of the effective area, and the loss function of the batch is calculated to train the basic classification model.

[0029] A storage medium stores a computer program, wherein the computer program is configured to execute any of the above methods when running.

[0030] An electronic device comprises a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute any of the above methods.

[0031] Compared with the existing methods, the present invention estimates the effective area in the feature space to give different weight losses to different categories, thereby better dividing the feature space. Specifically, first, to the best of the present invention's knowledge, the present invention is the first to discuss the shortcomings of the reweighting method based on category frequency, and proposes that the effective area of ​​the feature space is more effective for long-tail classification. Second, in order to estimate the effective area of ​​the feature space, the present invention derives a simple parameter-free formula based on the correlation between all samples in a category to estimate the effective area, and the formula can be adaptively updated as the training process progresses. Third, by taking the inverse of the effective area as the weight of each category, experiments show that a simple improved cross entropy loss can greatly improve the performance of long-tail classification. Fourth, the method of the present invention is not only effective in long-tail datasets, but also effective on balanced datasets, which shows that even on balanced datasets, there is a long-tail distribution form of the feature space. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 Flow chart of the method of the present invention.

[0033] Figure 2 Performance analysis of AREA and Decouple on many, medium, few, and all samples of the ImagNet-LT dataset.

[0034] Figure 3 The effective area of ​​different epochs of the dataset CIFAR-100-LT under different λ.

[0035] Figure 4 The effective area of ​​CIFAR-100-LT and balanced CIFAR-100 at different epochs under different λ.

[0036] Figure 5The effective area of ​​CIFAR-10-LT and balanced CIFAR-10 at different epochs under different λ. DETAILED DESCRIPTION

[0037] In order to make the above features and advantages of the present invention more obvious and easy to understand, the technical solution of the present invention is further described below through specific embodiments.

[0038] like Figure 1 As shown in FIG. 1 , the adaptive reweighting method based on the effective area of ​​the feature space (AREA) proposed in the present invention is divided into two stages. In the first stage, the present invention uses all training samples to train the feature extractor and the classifier to obtain a basic classification model. In the second stage, the present invention first uses the basic classification model to extract features from the training samples, and then calculates the correlation coefficient between any two samples in the same category, thereby obtaining a matrix R composed of the correlation coefficients between samples in the category. Then, the formula 1 / a is used to calculate the correlation coefficient between the two samples. T Ra calculates the effective area, a is a vector composed of n 1 / n numbers, n represents the number of training samples corresponding to the category, so the present invention can obtain the effective area of ​​each category through this formula. Finally, the present invention can use the inverse of the effective area to re-weight the loss of each category. Two points worth noting: First, the weights of the present invention can be adaptively adjusted. Since the features are optimized during the training process, the weights will also be updated accordingly. Second, the method of the present invention is parameter-free and weights different categories from a completely new perspective, so that a fairer classification model is obtained in the feature space.

[0039] (I) Adaptive effective area:

[0040] The present invention will first analyze the variance of a category when considering the sample distribution. Then, based on the assumption of filling the feature space with tangent samples, the present invention can obtain a compact variance. Finally, based on the full analysis of the correlation between any two samples in each category, a simple formula based on the effective area of ​​the feature space is derived.

[0041] 1.1 Distribution-based feature space: Given a long-tailed distribution dataset where y i ∈L={l 1 , l 2 , ..., l C}, where C is the total number of categories. For category l c A sample x in i , the feature f of the sample is obtained through the feature extractor i =F(xi ,θ f ), where the classifier is represented by H, θ f Represents the parameters of the feature extractor.

[0042] The feature space of a category is highly correlated with the feature distribution of samples of that category, so the present invention can use the variance of the category to measure the distribution of samples. c Class, if N c The samples are independent and identically distributed, that is, they have no correlation in the feature space. In this article, the present invention assumes that the samples are independent but identically distributed random variables. Then the variance of the class center of all sample random variables of category c can be calculated as shown in formula (1):

[0043]

[0044] Where c∈[1,C],N c Yes c The number of samples in the class, f i Indicates l c The characteristic of the i-th sample in , σ c Representative c The sample feature variance corresponding to the category. However, samples in the real world basically have a certain correlation. In this case, the calculation of the variance can be modified as shown in formula (2):

[0045]

[0046] Where N c Yes c The number of samples in the class, a c Represents the first c Category N c A vector of dimension, vector a c Each of the Right now Co c It represents l c The covariance matrix of all samples in the class, Co c Each diagonal element in is Yes c The variance of the class. and All belong to l c The characteristic distribution of the class, so and are equal to μ c , then for each off-diagonal element, the present invention can be defined as shown in formula (3):

[0047]

[0048] where μ c Yes c The class center of a class can be calculated as follows:

[0049]

[0050] Next, the present invention will recall the Pearson correlation coefficient used to reflect the linear correlation between two random variables X1 and X2. In this article, the present invention uses React two samples f i and f j The correlation between them can be calculated as follows:

[0051]

[0052] According to formula (5), the present invention can deduce sample f i and f j The correlation between Since the sample f i and f j are from the same distribution, so and are equal to σ c Through the above analysis, the present invention can obtain Co i,j As shown in formula (6):

[0053]

[0054] The present invention uses R c To represent sample-based The correlation matrix of i All of them have variance The distribution of, according to formula (6), the present invention can obtain the covariance matrix Co is the correlation matrix R c of Based on this, as described in formula (2), the present invention redefines as follows:

[0055]

[0056] 1.2 Effective area based on feature space: Inspired by formula (1), the present invention can obtain When there is a high correlation between samples, the effective area based on the feature may expand or compress. Assuming that the actual area of ​​the feature space is Then the present invention can be used tangent samples to fill the feature space. In this case, the variance of the related samples can be calculated using the following formula:

[0057]

[0058] This formula is reasonable. If the samples are completely uncorrelated, then So Degenerates into When there is a positive correlation between samples, Cov(f i , f j )>0, according to formula (2) we get Similarly in formula (8), when When there is When the samples are negatively correlated, Cov(f i , f j )<0, according to formula (2) Similarly in formula (8), when When there is That is, the compact form of formula (8) based on the effective area variance is reasonable. For example, if there are only two samples in a category, formula (13) can be degenerated to N eff =2 / (1+ρ). If the correlation between two samples is ρ=0.5, then the effective area is 4 / 3. If the correlation between two samples is ρ=-0.5, then the effective area is 4. If the correlation between two samples is ρ=0, then the effective area is 2, which is equal to the number of samples.

[0059] Through the above derivation, the present invention can combine formula (7) and formula (8) to obtain formula (9):

[0060]

[0061] Therefore, through formula (9), the present invention can deduce the effective area as:

[0062]

[0063] (ii) Training:

[0064] The effective area derived by the present invention is used for weighted operation on loss. The present invention will describe in detail the two stages of the training process. In the first stage, the present invention uses the original data to train the basic classification model through the standard cross entropy loss; in the second stage, the present invention assigns different weights of loss to different categories according to the obtained effective area. Below, the present invention will describe the second training stage in detail.

[0065] 2.1 Weighted loss function: In order to achieve balanced training, the present invention uses a weighted loss function, and the calculation formula is as follows:

[0066]

[0067] where z i =H(f i ,θ h ) is the predicted score of the category, w c Is category l c The corresponding weight, w c With l c The effective area of ​​the class is inversely proportional, |C| represents the number of data set categories and can be calculated according to the following formula:

[0068]

[0069] 2.2 Approximate effective area: Ideally, the effective area use Calculate, where R c for l c The correlation coefficient matrix of all samples in the class. However, in the implementation, R is calculated for all samples. c It is difficult to estimate accurately. Since long-tail data sets are usually large-scale, it is complicated to calculate too many large-scale correlation coefficient matrices. In order to simplify the calculation, the present invention calculates the effective area based on batch as shown in formula (13):

[0070]

[0071] where a cbi =1 / N cb , N cb Indicates l c The number of samples corresponding to the class when batch = b. Then, the present invention adds up the effective areas of all batches to obtain the effective area of ​​each class:

[0072]

[0073] Wherein, |B| represents the number of batches into which an epoch is divided during the training process. Therefore, as long as the present invention calculates the effective area corresponding to each batch Then the present invention can estimate the effective area of ​​each category through formula (14).

[0074] 2.3 Correlation coefficient matrix: In the task of the present invention, in order to obtain the effective area, the present invention needs to calculate the correlation coefficient between any two samples in the lc class to obtain l c The correlation coefficient matrix corresponding to each class. Because each sample fi is a d-dimensional vector, then each item in the matrix It can be calculated according to the following formula:

[0075]

[0076] Here, the present invention can recall that the cosine distance between two samples can be calculated according to the following formula (16):

[0077]

[0078] By comparing formula (15) and (16), the present invention can find that for f in formula (16) i ,f j After centralization, it can be equivalent to formula (15). Therefore, R cb Each item in It can be seen as the centralized {f i -μ c} and {f j -μ c}, which is easy to implement and can be calculated according to the following formula (17):

[0079]

[0080] 2.4 Online training algorithm: According to equation (17), the present invention can calculate each class l in each batch c The corresponding correlation coefficient matrix R cb , when R for each class is obtained cb , the present invention can calculate the effective area of ​​each category according to equation (13), and then at the end of each epoch, the present invention can add the effective areas of each category in each batch to obtain the effective area of ​​the total category. More specific details are as follows Figure 1 shown.

[0081] In summary, there are many methods for processing long-tail distributed data, among which reweighting is one of the simplest and most effective methods. Traditionally, researchers have tried to find a complex and subtle relationship between the number of samples and the loss weight to achieve fair training. However, the method of the present invention explores category weights from a new perspective of evenly partitioning the feature space. By fully considering the correlation of samples in a specific category, the present invention derives a simple, parameter-free method to measure the effective area of ​​the feature space. Then, compared with the cost-sensitive function method, a simple design of weights inversely proportional to the effective area can achieve significant improvements. In addition to the concise form and high performance, the method of the present invention has two other advantages: First, since the correlation of samples is extracted using features, the effective area and the corresponding weights can be adaptively updated at different training stages. Second, the method of the present invention is easy to implement and only requires simple modifications to the standard training process.

[0082] Specifically:

[0083] 1. The present invention proposes an adaptive reweighting method based on the effective area of ​​the feature space. This method calculates the effective area of ​​each category in the feature space by deriving a parameter-free effective area estimation formula. Simply using the inverse of the effective area as the weight of each category can better divide the category space.

[0084] 2. The present invention has no parameters and is simple and lightweight. In addition, the effective area will change adaptively with the training process and can be easily applied to other fields.

[0085] 3. The method of the present invention shows good performance on long-tail datasets (such as CIFAR-10 / 100-LT, ImageNet-LT and iNaturelist2018). In addition, it also has a good improvement effect on the balanced dataset CIFAR-10 / 100.

[0086] Effects of the Invention

[0087] Datasets: The present invention conducts a large number of experiments on large-scale long-tail datasets to evaluate the effect of the reasoning-based implicit data augmentation method (RISDA). The model of the present invention is trained and tested on the datasets CIFAR-10 / 100-LT, ImageNet-LT, iNaturalist 2018, and balanced CIFAR-10 / 100. Wherein (1) CIFAR-10 / 100 is a balanced dataset containing 50,000 training images and 10,000 validation images. In the experiments of the present invention, the imbalance factor λ, that is, the ratio of the number of samples of the most frequent and least frequent classes, is used to generate different training sets for CIFAR-10 / 100-LT. By setting different values ​​of λ in {200, 100, 50, 20, 10}, five long-tail training sets and the original balanced training set CIFAR-10 / 100 can be obtained. (2) The ImageNet-LT dataset is constructed by deleting some training samples in the ImageNet dataset. It has 115,846 training images, 20,000 validation images, and 50,000 test images. The class with the largest number of samples in the training set contains 1,280 samples, and the class with the smallest number of samples has only 5 samples. (3) iNaturalist 2018 is a large-scale species dataset with an imbalance factor of 500 and a severely unbalanced sample distribution. It contains 437,513 training images and 24,426 validation images from 8,142 classes.

[0088] Evaluation index: This invention adopts the standard index commonly used in classification: accuracy as the evaluation index, which represents the ratio of image pairs in all data sets, and the larger the value, the better.

[0089] Experimental results: Under the same settings and data partitioning, the proposed AREA method is compared with some of the best existing reweighting methods to verify its effectiveness. Tables 1, 2, and 3 show the experimental results on the datasets CIFAR-10 / 100-LT, ImageNet-LT, and iNaturalist 2018, respectively. Table 4 shows the experimental results on the balanced dataset CIFAR-10 / 100.

[0090] As shown in Table 1, the present invention can draw the following conclusions: First, the performance of directly using cross entropy loss to train long-tail data is the worst, and re-weighting loss functions can alleviate the impact of the long-tail problem because these methods can achieve relatively fair training by paying more attention to the tail categories. Secondly, compared with the best re-weighted method IB, the method of the present invention improves the accuracy of CIFAR-10-LT and CIFAR-100-LT by 1.03%, 0.62%, 0.98%, 0.19%, 0.46% and 6.15%, 5.83%, 6.03%, 4.12%, 3.72%, respectively, which illustrates the effectiveness of the method of the present invention. Finally, compared with CIFAR-10-LT, the improvement of CIFAR-100-LT is more obvious, and the imbalance degree of the corresponding tail samples of CIFAR-100-LT is higher than that of CIFAR-10-LT. Therefore, this shows that the method of the present invention is more suitable for processing large-scale long-tail data.

[0091]

[0092]

[0093] Table 1 Accuracy of ResNet-32 on CIFAR-10 / 100-LT under different imbalance factors

[0094] From the experimental results in Table 2, it can be seen that the accuracy of the method AREA of the present invention on ImageNet-LT is improved to 49.53%, which achieves the best performance among the reweighted methods. In addition, compared with the class-balanced loss, the method of the present invention is improved by 8.68%, which shows that the method of the present invention can calculate better weights for the loss, thereby obtaining a more unbiased classifier. In order to analyze the results in more detail, the present invention further demonstrates the accuracy of ImageNet-LT on three groups: Many-shot, Medium-shot, and Few-shot: Many-shot means classes containing more than 100 images, Medium-shot means classes containing 20-100 images, and Few-shot means classes containing less than 20 images. Figure 2 The results are shown. The method of the present invention is compared with the Decoupling method which is also a two-stage method. It can be seen that the method of the present invention can achieve great improvements on all three groups while minimizing the performance of the head class.

[0095]

[0096] Table 2 Performance comparison of different methods on the ImageNet-LT dataset

[0097] As shown in Table 3, the accuracy of the proposed method in iNaturalist 2018 is improved to 68.36%, which is the best performance among the reweighted methods. Compared with Class-balanced, the proposed method is improved by 7.24%. More importantly, the accuracy of the proposed method is 2.97% higher than the current best reweighted method IB, which also shows that the proposed method can obtain a more unbiased classifier.

[0098]

[0099] Table 3 Performance comparison of different methods on the iNaturalist 2018 dataset

[0100] As shown in Table 4, we can see that even for balanced CIFAR-10 / 100, the method of the present invention can achieve an improvement of {0.23%, 0.90%} respectively compared with the cross entropy. Inevitably, a statistically balanced distribution may not mean that the space of each category is equal, which is proved by Table 4.

[0101]

[0102] Table 4 Experimental results on the balanced CIFAR-10 / 100 dataset

[0103] Statistical distribution and spatial distribution analysis: To show the difference between statistical distribution and feature space distribution, we calculated the effective area of ​​the feature space on CIFAR-100-LT and recorded several epochs when λ = {100, 20, 1}. Figure 4 As shown, the present invention can obtain the following observation results:

[0104] First, the size of the effective area can be larger, lower, or equal to the original number of samples. When the imbalance ratio is λ=100, "beaver" has 415 samples. However, its effective area ranges from 449 to 427, which is larger than the original 415. When the imbalance ratio is λ=20 and epoch=160, the effective area of ​​the "house" category is 157.18, which is lower than its original number of samples of 163. Second, the feature space distribution varies in different training epochs, that is, the weights of the present invention can be adaptively adjusted. This is because the feature representation is optimized as the training process proceeds. Accordingly, the correlations between samples are updated, resulting in a variable feature space-based effective area. Third, the effective area of ​​the feature space of a statistically balanced dataset is unbalanced. The present invention reorders the categories according to the size of the effective area and finds that the balanced dataset presents a long-tail distribution. As Figure 3As shown, the effective area of ​​"rose" (532.54) is larger than its sample number 500. The effective area of ​​"beaver" (495.31) is smaller than its sample number 500. In addition, the effective areas of most categories are larger than their statistical sample numbers. The present invention believes that this is because each category in the balanced dataset has enough samples with rich diversity, such as more backgrounds, angles, poses, etc. For more results on the effective area of ​​CIFAR-10 / 100-LT, please see Figure 4 , 5.

[0105] In summary, AREA has obvious advantages over other best methods, indicating that the effective area-based reweighting proposed in the present invention is more suitable for processing extremely unbalanced data sets. Even for balanced data sets, there is still imbalance in the feature space, and the method of the present invention can also be improved on balanced data sets.

[0106] The above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. A person skilled in the art may modify or replace the technical solutions of the present invention with equivalents. The protection scope of the present invention shall be subject to the claims.

Claims

1. An adaptive weighted long-tail classification method based on effective area of ​​feature space, the steps of which include: Train the feature extractor and classifier on the distribution of image data to obtain a basic classification model; Extracting feature representations of the image data based on the trained feature extractor, and constructing a correlation coefficient matrix of each category according to correlation coefficients between the feature representations in the category; Based on the correlation coefficient matrix, calculate the effective area of ​​each category in the feature space And assign weight ω to each category through the effective area c , train the basic classification model to obtain a classification model; wherein the weight |C| represents the number of categories, Indicates category l c The effective area in the feature space is calculated based on the correlation coefficient matrix to calculate the effective area of ​​each category in the feature space. include: Define category l c The variance of the class center of all sample random variables N c Is category l c The number of samples in They are respectively c 1, 2, ..., N c The characteristics of the samples, Is category l c The variance of c Representative category c N c A vector of dimension R c Indicates category l c The correlation coefficient matrix of Due to the use of In the case where tangent samples are used to fill the feature space, the category l c The variance of the class center of all sample random variables Derive the category l c The effective area in feature space Based on the classification model, a classification result of the data to be detected is obtained.

2. The method according to claim 1, characterized in that The method for training a feature extractor and a classifier on the distribution of image data includes: training based on cross entropy loss.

3. The method according to claim 1, characterized in that The step of constructing a correlation coefficient matrix of each category according to the correlation coefficients between the feature representations in the category includes: For category l c , calculate the covariance of any two feature representations; Calculation Category c The variance of Based on the covariance and category l c The variance of , obtains the correlation coefficient between any two feature representations; Based on the correlation coefficients, construct the correlation coefficient matrix; or, Calculation Category c The class center μ c ; By taking the centralized {f i -μ c } and {f j -μ c } as each item of the correlation coefficient matrix to obtain the correlation coefficient matrix of this category, where f i 、f j Divided into image data x i 、x j The feature representation of .

4. The method according to claim 1, characterized in that The loss function for training the basic classification model N represents the number of image data, Represents image data x i In category l c The prediction score on .

5. The method according to claim 1, characterized in that When training the basic classification model, the effective area of ​​each category is obtained based on the sum of the effective areas of all batches in the feature space, the weight of each category is calculated based on the inverse of the effective area, and the loss function of the batch is calculated to train the basic classification model.

6. A storage medium storing a computer program, wherein: The computer program is configured to execute the method according to any one of claims 1 to 5 when executed.

7. An electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 5.

Images