Classification method based on conversion from majority class to minority class under unbalanced data set

Abstract

The invention discloses a classification method based on conversion from majority classes to minority classes under an unbalanced data set. The method comprises the following steps: preprocessing overall training data; selecting a certain number of samples as partners from the majority classes for each minority class sample, and determining the value range of the number; selecting proper samples from the majority classes to form a new sample set; changing labels of the samples in the set; carrying out lifting training on the weak classifier by adopting a new loss function pair forward addition model, wherein in the training process, the final classifier distribution is solved, and the weights of the optimal base classifier and the modified base classifier are solved for the target function in each step; and performing related pre-training by using a classifier, and determining the final number of the majority classes converted into the minority classes. The method not only can be applied to image classification and image recognition, but also can be applied to natural language processing and other scenes needing classification.

Description

technical field [0001] The invention relates to a classification method, more specifically, to a classification method based on conversion of majority class into minority class under an unbalanced data set. Background technique [0002] In image classification or recognition, many algorithms have a basic assumption that the data distribution is uniform. When we apply these algorithms directly to real data, such as medical treatment and fraud, most of the cases cannot achieve ideal results. Because the actual data is often very unevenly distributed, there will be a "long tail phenomenon", which is the problem of unbalanced classification. Generally speaking, a data set called an imbalanced data set needs to meet two conditions: the imbalance of the number of categories and the imbalance of misclassification costs. Taking the binary classification problem as an example, assuming that the number of samples of the negative class is much larger than that of the positive class, ...

AI Technical Summary

Problems solved by technology

In the article "Neighborhood-based undersampling approach for handling imbalanced and overlapped data", it is clearly pointed out that although the improvement method at the data level is simple in thinking, it increases the computational overhead; although the improvement method at the algorithm level is faster, it lacks flexibility and is difficult to implement in different environments. Migrating between datasets

Images