Two-stage resampling method for unbalanced data

Abstract

The invention discloses a two-stage resampling method for unbalanced data. The method comprises the following specific steps: firstly, increasing the number of minority class samples by adopting an SMOTEENN method to enable a data set to tend to be balanced; then introducing positive samples and labels sampled by the SMOTEENN as the input of a generative network, so as to enable a CGAN to be able to fully learn distribution characteristics of the positive samples in the training process; then, based on the generative network in the CGAN, carrying out amplification on minority class samples to synthesize a new balanced data set. The method provided by the invention is applied to a plurality of shared standard unbalanced data set classification experiments, and experiment results show that compared with other classical unbalanced data set processing methods, the method provided by the invention has the advantages that the distribution of generated samples is more reasonable, and the performance on a classifier is more advantageous.

Description

technical field [0001] The invention belongs to the technical field of classification methods for unbalanced data sets in machine learning, and relates to a preprocessing method for unbalanced data classification based on conditional generative confrontation networks and SMOTEENN. Background technique [0002] As the core technology in the field of artificial intelligence, machine learning is widely used in big data analysis. By establishing classification or regression models for massive and complex data, valuable information and laws can be learned from them. Traditional classification methods are usually based on a basic assumption that the number of samples in each category in the data set is consistent or equal and the cost of misclassification is equal. However, the data that can be collected in practical application scenarios often has the problem of data imbalance. Under the condition of data imbalance, the traditional classification learning method with the overall ...

AI Technical Summary

Problems solved by technology

[0008] The purpose of the present invention is: traditional sampling methods often produce more overlapping samples when constructing a balanced data set, and CGAN, which is more advantageous in dealing with unbalanced data, often cannot fully learn its distribution characteristics due to the limitation of the number of positive samples It is difficult to generate high-quality synthetic samples, and a two-stage resampling method for imbalanced data is proposed

Images