Chinese short text classification method based on characteristic extension

Abstract

The invention provides a Chinese short text classification method based on characteristic extension, and the method comprises the following steps that (1) a background knowledge base is established: the two-tuples of feature words which meet a certain constraint condition are dug from a long text corpus with category marks to form the background knowledge base; (2) short text which is trained in a centralized way is extended: extension words are added to the short text which is trained in a centralized way according to a certain extension rule according to the two-tuples in the background knowledge base; (3) a classification model is built: a (shared virtual memory) SVM classification model is established through an extended short text training set; (4) the short text to be classified is extended: the extension words are added to the short text to be classified according to a certain extension rule according to the two-tuples in the background knowledge base and the feature space of the classification model; and (5) a classification result is generated: the classification result is generated through the classification model and the extended short text. According to the Chinese short text classification method based on characteristic extension, the features of the short text are enriched through the long text corpus, so that the accuracy and the recall rate in the classification of the short text are improved.

Description

technical field [0001] The invention relates to the technical field of text classification systems, in particular to a method for classifying Chinese short texts based on feature expansion. Background technique [0002] According to statistics, about 80% of electronic information data exists in the form of unstructured text files. On the Internet, not only text data is the most common form of data storage, but also the search for data such as video, audio, and pictures has text data associated with it. [0003] Text classification is a key technology for processing and organizing massive text data, which can effectively solve the problem of information clutter, and facilitate users to accurately locate the required information and distribute information. Traditional text classification systems mainly use classification methods such as KNN and SVM, which can achieve good classification results in long text classification applications. [0004] With the continuous developmen...

AI Technical Summary

Problems solved by technology

[0005] Since the features of short texts are discrete and short in length, traditional text classification methods cannot achieve comparable classification results to long text corpora when directly applied to short text corpora.

Images