Unbalanced data processing method based on integrated feature selection in machine learning

Abstract

According to the invention, an unbalanced data processing method based on integrated feature selection in machine learning is designed; the method comprises the following steps: constructing a majority class sample and a minority class sample into a plurality of bagged subsets by adopting a method of combining a Bagging method and a method of Synthetic minority oversampling technique (SMOTE); andthen screening out a feature subset from each bagged subset by utilizing a feature selection algorithm based on correlation measurement, setting a threshold value to obtain a feature set, and finallyconstructing a classification model by utilizing a machine learning algorithm to perform classification research.

Description

technical field [0001] The invention relates to the field of data processing, and specifically designs an unbalanced data processing method based on integrated feature selection in machine learning. Background technique [0002] With the rapid development of mobile Internet technology, data collection and data transmission have become more convenient and faster. The difficulty of information technology is no longer in data acquisition and transmission, but how to efficiently process and utilize known data, and dig out all the information in it. The existence of valuable information has promoted the rapid development of data mining. But in the research process of data mining and machine learning, we often encounter a serious potential challenge, that is how to deal with the data of "unbalanced category". [0003] In the process of classifying unbalanced data sets, the performance of traditional machine learning classification algorithms is greatly hindered. The current proce...

AI Technical Summary

Problems solved by technology

[0004] The present invention designs an unbalanced data processing method based on integrated feature selection, which is used to solve the problem of unbalanced data in a given data set. In order to achieve this purpose, the described processing method uses integrated feature selection to solve It solves the problem of reduced training efficiency caused by redundant features and biased decision boundaries caused by unbalanced data, improving model accuracy and training efficiency

[0006] Different from the existing processing methods, the beneficial effects of the present invention are: the current solution will not only greatly increase the computational complexity, easily lead to model over-fitting, but also lead to low accuracy of the model. In addition, the current big data era The unbalanced data is more complex and changeable. The processing method of this design can be well applied to the unbalanced data at the current stage, achieving a better ability to deal with unbalanced data, improving the accuracy of the model and training efficiency, and practicality. higher

Images