Re-sampling and cost-sensitive learning integrated unbalanced data integration and classification method

Abstract

The invention discloses an unbalanced data integration classification method that combines resampling technology and cost-sensitive learning, relates to the field of artificial intelligence integrated learning, and mainly solves the problem of unbalanced data classification using complete data information in the prior art. The method The steps are: (1) input the training data set; (2) calculate the relative density of the sample space distribution; (3) resample to generate multiple subsets and train the basic classifier; (4) calculate the similarity matrix of the test sample; (5) ) using multi-objective optimization and integration to obtain prior results; (6) performing cost-sensitive learning prediction on the test set; (7) using KL divergence to optimize and fuse the results. The method designs a new sampling method to solve the problem of unbalanced data distribution; uses a method combining resampling technology and cost-sensitive learning to solve the problem of incomplete information; and makes full use of the data information of the test set itself to improve integration performance of the classifier.

Description

technical field [0001] The invention relates to the field of computer artificial intelligence, in particular to an integrated classification method combining unbalanced data resampling technology and cost-sensitive learning simultaneously. Background technique [0002] Most of the standard machine algorithms proposed so far are designed on the assumption of balanced data distribution or equal error cost, so they are not suitable for processing data with unbalanced class distribution. If the standard learning algorithm is directly applied to the unbalanced data, the classification rules summarized for the class with a small sample size are less and less reliable than those with a large sample size. [0003] The traditional unbalanced learning classification methods are mainly classified into two categories, using data-level resampling technology to correct the imbalance of training samples and the design of cost-sensitive functions at the algorithm level to correct the advers...

AI Technical Summary

Problems solved by technology

[0004] The current traditional unbalanced learning has limitations: First, a large number of studies and experiments have proved that the method based on sparse sampling in resampling technology can improve the classification performance better than the method of oversampling, but the method of sparse sampling will lose part of the original data information, which is not all redundant information

Second, the effect of cost-sensitive learning is usually not as good as that of classification after resampling, and it is extremely sensitive to outliers and noise points, which is only suitable for training standard imbalanced data

Third, when the traditional ensemble learning method is used to deal with unbalanced data, it usually only focuses on optimizing the steps of resampling to generate a new more balanced data set, and does not pay attention to the information between different test samples.

Images