Feature selection method and device of high-dimension data

Abstract

The invention discloses a feature selection method and device of high-dimension data. The method comprises the following steps: obtaining an original data set to be processed, wherein the original data set comprises a feature set, a plurality of samples and a category set, and the category set comprises the category of each sample; calculating to obtain a MIC (Maximum Information Coefficient) between each feature in the feature set and the category set, and the redundant value of each feature and a selected feature subset; and according to the MIC and the redundant value, obtaining the effective value of each feature, and selecting the feature subset from the feature set according to the effective value. The MIC is introduced into feature selection, and the feature is effectively evaluated on the basis of the MIC so as to select features according to the effective value generated by evaluation. Compared with the prior art, the feature selection method can effectively improve accuracy for high-dimension data feature selection.

Description

technical field [0001] The invention relates to the technical field of data mining, in particular to a feature selection method and device for high-dimensional data. Background technique [0002] The rapidly developing information society generates massive amounts of data every day, and how to quickly discover useful information from these data has become an urgent problem to be solved. Researchers have tackled this problem from the perspective of machine learning models and have made remarkable progress. However, it is increasingly difficult for high-complexity models and high-dimensional feature spaces to meet the urgent requirements of big data applications, and there are often a lot of useless information in feature spaces. Only by adopting an appropriate feature selection method can effective features be obtained from massive data, thereby improving the efficiency and accuracy of machine learning models in processing data; at the same time, feature selection can also p...

AI Technical Summary

Problems solved by technology

Although MIC is very effective in variable measurement, it can only measure the correlation and redundancy between single variables. Therefore, this paper proposes a new measurement mMIC (effective value) and applies it to the Markov blanket condition to solve the current problem. There is a problem of low accuracy of feature selection due to the difficulty of applying redundancy between features and feature subsets in high-dimensional datasets

Images