Text classification feature selection method and application thereof to biomedical text classification

Abstract

The present invention provides a text classification feature selection method and application thereof to biomedical text classification. Local context similarity calculation based on shallow grammatical analysis is adopted, a feature selection algorithm is proposed, and based on measurement of local context similarity of feature words, features are found whether to exist in a certain specific modes so as to measure importance of the features. Moreover, a feature selection method (LLFilter method) based on local context similarity is adopted and by filtering of the features, a sample is enabled to obtain the best classification effect, i.e. inter-class dispersion of the sample obtained after feature filtering reaches the highest, and within-class dispersion reaches the lowest, so that the ability of distinguishing classes is improved. According to the text classification feature selection method and the application thereof to biomedical text classification, which are provided by the present invention, mainly in a biomedical text classification task, local context information in a text is utilized to automatically perform feature importance sorting, so that a feature set is optimized, a dimension of a feature space is reduced, and performance of text classification can be effectively improved.

Description

technical field [0001] The invention relates to a text classification feature selection method, in particular to a text classification feature selection method based on local context similarity, and belongs to the technical field of big data mining. Background technique [0002] With the advent of the information age and the rapid development of information technology, the Internet provides people with extremely rich information resources, resulting in an exponential growth in the amount of information. In order to effectively manage and utilize the information, content-based information retrieval and data mining have gradually become areas of concern. Among them, the realization of automatic text classification has become a key technology with practical value, especially now that manual classification is powerless in the face of massive texts, automatic text classification is particularly important. Text classification (Text Classification, referred to as TC) technology is...

AI Technical Summary

Problems solved by technology

[0008] When most of the Filter methods evaluate irrelevant features, their evaluation functions are based on the assumption that each feature is isolated, without considering the specific patterns that may exist between feature words

Images