Text classification method for different subject topics

Abstract

The invention is suitable for the technical field of data pre-processing, and provides a text classification method for different subject topics. The method comprises the following steps of: A, performing word selection on each subject by using a chi-square test to form a feature vocabulary of the subject; B, performing subject classification on the selected feature words by using a Naive Bayes model; and C, performing secondary classification on first two structures with maximum probabilities given in the Naive Bayes model classification by using a support vector machine to obtain a first result. Through two-time classification, the average accuracy of classification is improved. The text classification method for different subject topics is simple to realize and operate, convenient and accurate to use and more accurate in classification between the subjects, and effectively improves the accuracy of classification between the adjacent subjects.

Description

technical field [0001] The invention relates to data preprocessing technology, in particular to a method for classifying texts of different subjects. Background technique [0002] With the explosive growth of text information on the Internet, the demand for text processing is becoming more and more urgent, and the precision and accuracy required are also getting higher and higher, especially in the fields of document classification and information retrieval, it is often necessary to process large batches of documents are automatically classified. [0003] The current text classification method mainly includes three links, namely text representation, feature extraction and text classification. Generally speaking, the main difference between different text classification methods lies in how to represent the text. In terms of text representation, there are mainly two text classification methods based on dictionary vectors and deep learning. The former directly represents the t...

AI Technical Summary

Problems solved by technology

[0004] The current text classification method has a high accuracy rate when dealing with texts with obvious characteristics and large differences between categories, but the effect will be reduced when dealing with texts with a certain degree of similarity. Topics, namely Mathematics, Language, Foreign Languages, Objects, Chemistry, Biology, Politics, History, and Geography are examples. Among them, it is easier to classify science and liberal arts, but there is a certain degree of similarity between science and liberal arts.

In the method based on dictionary vectors, statistical learning methods are generally used when selecting feature words. Generally, only the information of words is considered in statistics, and the association between words is ignored; After words are represented as vectors, although the vectors contain the association information between words, when using word vectors to represent the entire text, it is difficult to find a unified feature input classifier due to the large variation in the length of different texts. In the scheme of using deep learning, the length of the text is fixed, which will inevitably lead to the loss of information

Images