A Feature Selection Method for Text Classification

Abstract

The invention provides a text classification feature selecting method capable of reducing the characteristic dimension and the classification complexity and improving the classification accuracy. The method comprises the following steps that a feature set S and a target class C are obtained, the relevancy Rc(x(i)) between each feature x(i) in the feature set S and the target class C is calculated, and sort descending is conducted on the feature set S according to the size of the relevancy Rc(x(i)); the redundancy Rx and the synergy degree Sx between every two features in the feature set S are calculated, and the sensitivity Sen of each feature is calculated with the combination of the relevancy Rc(x(i)) between each feature and the target class, the sensitivities Sen are compared with a preset threshold th, and the feature set S is divided into a candidate set Ssel and an excluding set Sexc with the combination of the sort descending result of the feature set S according to the threshold th; the sensitivities Sen between the features in the candidate set Ssel and the features in the excluding set Sexc are calculated, the sensitivities Sen are compared with the preset threshold th, and the candidate set Ssel and the excluding set Sexc are adjusted according to the threshold th. The text classification feature selecting method is suitable for the field of machine learning text classification.

Description

technical field [0001] The invention relates to the field of machine learning text classification, in particular to a text classification feature selection method. Background technique [0002] With the continuous expansion of the scale of the Internet, the information resources gathered in the Internet are also increasing. In order to effectively manage and utilize these information resources conveniently, content-based information retrieval and data mining have always been concerned. Text classification technology is an important basis for information retrieval and text data mining. Its main task is to distinguish unknown categories of words and documents as one or more of the predetermined categories. However, the two characteristics of large number of training samples and high vector dimension determine that text classification is a machine learning problem with high computational time and space complexity. Therefore, we need to perform feature selection to reduce the ...

AI Technical Summary

Problems solved by technology

[0004] The technical problem to be solved by the present invention is to provide a text classification feature selection method to solve the problems of high feature dimension or low classification accuracy existing in the prior art

Images