Statistic text classification system and statistic text classification method based on term frequency-inverse document frequency (TF*IDF) algorithm

Abstract

The invention relates to a statistic text classification method based on a term frequency-inverse document frequency (TF*IDF) algorithm, which proposes a novel characteristic vector weight method (TF*IDF*CIV), a variable, namely concept information amount (CIV) is introduced in the TF*IDF algorithm, the concept information amount of a characteristic vector serves as a variable to be considered in the calculating process of the characteristic vector weight. A formula of the algorithm is that sharing concept number sim (ci, C) is matched and equal concept number of a concept set ci corresponding to a characteristic item ti in a category concept set C. The shortage of the TF*IDF algorithm is overcome, and the TF*IDF algorithm is widely used for calculating the characteristic vector weight. The statistic text classification method cannot express relevance between characteristic items and ignores the influence of the relevance between characteristic items to the weight. Therefore, experimental results prove that the accuracy rate of a whole text classification system can be effectively improved by means of the novel method.

Description

technical field [0001] The invention relates to the field of computer science and technology, in particular to a new calculation method and device for feature vector weights oriented to text classification. Background technique [0002] With the rapid development and popularization of Internet technology and computer technology, a large amount of text information begins to exist in a computer-readable form, and computer automatic text classification technology emerges as the times require. At present, text classification technology is widely used in various research fields such as document indexing, bad information detection, topic recognition, automatic summarization, and intelligent information retrieval. [0003] Automatic classification research began in the late 1950s, and H.P. Luhn conducted pioneering research in this field. In 1961, Maron published the first paper on automatic classification, and then many famous intelligence scientists such as Sparck and Salton con...

AI Technical Summary

Problems solved by technology

[0006] The purpose of the present invention is to overcome the problem that the current TF*IDF algorithm does not consider the semantic similarity between words when calculating the weight of feature items, which causes the low accuracy of the text classifier based on the TF*IDF algorithm, and provides a Statistical Text Classification System and Method Based on TF*IDF Algorithm

Images