Classification method of unbalanced data

Abstract

The invention discloses a method for classifying unbalanced data, and belongs to the technical field of machine learning, the method comprises an active learning method and an oversampling method, the unbalanced data comprises marked data and unmarked data, and the method specifically comprises the following steps: preprocessing the marked data, and calculating distance features to obtain an initial training set; training the initial training set to obtain an initial classifier; calculating the uncertainty of the unmarked data by using the initial classifier; sorting the unmarked data according to the uncertainty, and manually marking the unmarked data to obtain a marked data set; performing probability oversampling on the marked data set to obtain a balanced data set; and training the balanced data set to obtain a classifier which is used for classifying unbalanced data. According to the unbalanced data classification method, active learning and an oversampling method are combined, so that the number of samples participating in training is reduced; meanwhile, it is guaranteed that the classifier has high classification precision for majority class data and minority class data.

Description

technical field [0001] The invention relates to a classification method of unbalanced data, which belongs to the field of machine learning. Background technique [0002] At present, the research on the problem of data imbalance is mainly carried out at the level of data preprocessing, feature level and classification algorithm level, so as to ensure that the classifier has high classification accuracy for both majority and minority class data. At the level of data preprocessing, the imbalance is reduced or eliminated by changing the sample distribution of the training set. The specific method is a series of undersampling and oversampling techniques; the imbalance of the number of samples at the feature level is often accompanied by the uneven distribution of feature attributes. Balance, use the feature selection method to select features with distinguishing characteristics, improve the classification accuracy of minority classes; at the level of classification algorithms, ac...

AI Technical Summary

Problems solved by technology

If a conventional algorithm is used to deal with this problem, the classification result will tend to be biased towards the majority class, resulting in the minority class being unable to be correctly identified

However, most traditional algorithms train classifiers based on maximizing the overall correct rate, so the influence of minority class samples will be ignored, resulting in misclassification of minority classes, thus affecting the classification results of traditional classifiers.

Images