Deep learning-based balanced sampling and modeling method for unbalanced data

Abstract

The invention relates to a deep learning-based balanced sampling and modeling method for unbalanced data. The method comprises the steps of extracting majority class and minority class sample sets ina data set, and performing counting respectively; and carrying out multi-time K-Means clustering on the majority class sample set to obtain R clustering results; performing clustering fusion on the Rclustering results by adopting a clustering fusion algorithm based on an incidence matrix, thereby obtaining a new majority class sample set; performing over-sampling on the minority class sample set,thereby obtaining a new minority class sample set; combining the obtained new majority class sample set and new minority class sample set, thereby forming a new data set with class balance; extracting an abstract feature of the new data set with the class balance, and adding the abstract feature as a new feature into a feature set of the new data set with the class balance, thereby forming a newfeature set; and training a DBN model by adopting the obtained new feature set, thereby obtaining an optimal DBN model. The method overcomes the shortcomings in a single processing method, has a better processing capability, and has relatively high accuracy.

Description

technical field [0001] The invention relates to a balanced sampling and modeling method of unbalanced data. In particular, it relates to a balanced sampling and modeling method for unbalanced data based on deep learning. Background technique [0002] For the problem of data imbalance, the current research directions are mainly divided into two types: research on data imbalance methods based on the data level and research on data imbalance methods based on the algorithm level. [0003] The data imbalance problem at the data level mainly adopts two methods: oversampling and undersampling. Oversampling is to increase the number of minority class samples so that the total number is balanced with the majority class samples, and the corresponding undersampling is to reduce the number of samples, or to extract certain samples from them so that the number is balanced with the minority class. [0004] A commonly used oversampling method is Adaptive Synthetic Sampling Approach (ADAS...

AI Technical Summary

Problems solved by technology

[0008] The technical problem to be solved by the present invention is to provide a balanced sampling and modeling method for unbalanced data based on deep learning, which can effectively solve the problem of poor classification performance for minority classes when ordinary classifiers process unbalanced medical data. Poor, which leads to the inaccurate problem of the established forecasting model

Images