High-dimensional data feature selection method based on filtering method and genetic algorithm

Abstract

The invention discloses a high-dimensional data feature selection method based on a filtering method and a genetic algorithm. Traditional feature selection methods are not suitable for high-dimensional and small-sample data because they have the limitation of high probability of falling into local optimization and deleting useful features. According to the invention, firstly, the maximum information coefficient is adopted to calculate the correlation between features of input data and class marks; then, the features are sorted in descending order according to values of the correlation, a threshold value is set, and features with weak correlation are deleted; and finally, the remaining features with strong correlation are subjected to random searching optimization by the genetic algorithm to obtain an optimal feature subset. The invention can effectively carry out feature selection on high-dimensional data and realize dimension reduction. The result of feature selection has important significance for sample class judgment, and when the method is applied to gene expression profile data, the selected features also have important biological significance.

Description

technical field [0001] The invention belongs to the technical field of data mining, and relates to a high-dimensional data feature selection method based on a filtering method and a genetic algorithm. Background technique [0002] The advancement of data collection and storage technology has made organizations accumulate massive amounts of data, and how to extract useful information from it has become a huge challenge at present. High-dimensional data generally has the characteristics of data sparsity and dimensionality disaster. The sparse nature of high-dimensional data, which is mostly zero and a few have values, makes it difficult to analyze and mine the data directly. As the data dimensions (attributes) increase, the amount of computation increases exponentially, resulting in the curse of dimensionality. Through effective feature selection on high-dimensional data, features that are effective for recognition or classification are selected, thereby simplifying calculat...

AI Technical Summary

Problems solved by technology

[0003] The method introduced above has limitations such as easy to fall into local optimum and high probability of deleting useful features, and is not suitable for feature selection of high-dimensional and small sample data.

Images