Two-stage combined file classification method based on probability subject

Abstract

The present invention relates to the field of natural language processing and pattern recognition, and discloses a two-level combined text classification method based on probabilistic subject words, and the first-level classification: based on the naive Bayesian classification method, the test text is classified by using the characteristics of the probabilistic subject words and the judgment of rejection conditions; Second-level classification: Based on the information gain feature extraction method, feature words are extracted to classify the test texts that were rejected by the first-level classification. The hierarchical combination method of the present invention classifies texts, and integrates the characteristics of different classifiers to correctly classify many texts in the first-level classification very quickly, greatly improving the efficiency of the text classification system, and providing good processing for the practical application of the text classification system Method; Considering the characteristics of the text, the probabilistic keyword is proposed. Under appropriate rejection conditions, the probabilistic keyword can complete a large number of text classification tasks with a high accuracy rate. Experiments have proved that the two-level combination of the present invention can greatly reduce time consumption and improve the classification accuracy rate of the system compared with the traditional single classification.

Description

technical field [0001] The invention relates to the technical fields of pattern recognition and natural language processing, and relates to a method for classifying serially combined texts based on probabilistic subject words. Background technique [0002] Text Categorization (Text Categorization) is one of the comprehensive applications of various natural language processing technologies. Realizing automatic computer classification of text can better help us organize and utilize the current vast text information. At the same time, text classification methods involve many basic problems in pattern recognition, such as classifier design problems, high-dimensional feature problems and so on. Therefore, the research on text classification technology has important practical value and theoretical significance. [0003] To measure the quality of a text classification method, two factors are generally considered. One is the correct rate of classification results, which is often t...

AI Technical Summary

Problems solved by technology

[0004] In order to solve the defect that it is difficult for a single classifier in the prior art to have obvious advantages in terms of accuracy and efficiency, the purpose of the present invention is to make up for the shortcomings of a single classifier, propose a combined classification method, and realize a probability-based theme Two-level Combination Text Classification Method of Words

Images