A multi-classification oriented unbalanced data preprocessing method and device and an apparatus

Abstract

The invention discloses a multi-classification oriented unbalanced data preprocessing method and device and an apparatus. The method comprises the following steps: receiving the final sample set sizeand the unbalanced ratio of the sample set, and obtaining the ideal sample number of each class; according to the number of ideal samples and the number of actual samples, judging the sample sets of minority classes and majority classes; for the samples in the sample set of a few classes, calculating the number of other class samples and a few class samples in the k-nearest neighbor to classify the samples; for the sample set of a few classes, performing deleting, saving, copying or synthesizing according to the marker of the sample set to obtain the final sample set of a few classes; For thesamples in most of the sample sets, calculating the number of the samples in the k-nearest neighbors and other samples to classify the samples. The samples in most class sample sets are deleted or saved according to the markers of the samples to obtain the final sample sets of most classes. The final sample set is generated. The invention enables the final sample set to effectively improve the accuracy of the multi-classification algorithm.

Description

technical field [0001] The present invention relates to the field of big data processing, in particular to a multi-category-oriented unbalanced data preprocessing method, device and equipment. Background technique [0002] With the continuous advancement of technology, including the improvement of Internet speed, the upgrading of mobile Internet, the continuous development of hardware technology, the rapid development of data acquisition technology, storage technology and processing technology, data is growing at an unprecedented rate, and we have entered the big data era. The characteristics of big data such as huge data volume (volume), high speed (velocity), variety (variety), and data uncertainty (veracity) make traditional data analysis and mining technologies encounter unprecedented challenges when applied to the field of big data. . [0003] Data classification is a basic algorithm in data analysis and mining, which has a wide range of applications and is also the b...

AI Technical Summary

Problems solved by technology

[0008] 2. A certain sample may be of different sample categories in different binary classification problems. For example, it is noise in one binary classification and needs to be deleted. It is an important boundary sample in another binary classification and needs to be retained. Use Existing methods cannot handle it effectively

[0009] In short, if the multi-classification problem is considered as multiple binary classification problems, the processing of samples cannot comprehensively consider the different situations in each category, and the accuracy of the multi-classification algorithm cannot be effectively improved.

Images