Class center vector text classification method based on dependency, word class and semantic dictionary

Abstract

The invention relates to text classification of natural language processing, and specifically relates to a class center vector text classification method based on dependency, word class and a semanticdictionary. To overcome the semantic defect of a feature selection algorithm based on statistics, the invention introduces the dependency, the semantic dictionary and the word class to optimize and cluster text features, provides an improved weight calculation formula, and further provides an improved class center vector text classification method. The text classification method of the inventionhas advantages of both high classification efficiency of a traditional class center vector method and high classification precision of a K-nearest neighbor algorithm, and can be widely used in variousclassification systems.

Description

technical field [0001] The invention relates to text classification in natural language processing, in particular to a class center vector text classification method based on dependency relationship, part of speech and semantic dictionary. Background technique [0002] With the rapid development of computer technology, especially in the context of the "Internet +" era, network information such as documents, pictures, audio and video has exploded exponentially, and a large number of electronic files exist in the form of electronic files in daily life. How to obtain the desired information from massive data is a hot and difficult point in current research, and text classification is one of the important research directions. [0003] Text classification is an important research direction in text processing technology, which began in the 1950s. It is a comprehensive technology integrating linguistics, mathematics, computer science and cognitive science. In the late 1950s, H.P. ...

AI Technical Summary

Problems solved by technology

Although the Bayesian algorithm is simple in principle and easy to implement, it is based on an assumption that only when the text data sets are independent of each other, the classification accuracy will be high, so it has certain limitations when used in text classification; K nearest neighbor algorithm The classification accuracy is very high, but the classification efficiency is very low. It has a relatively good classification effect in the face of small-scale corpus, but it will have a long classification time when encountering large-scale corpus; support vector machine due to its generalization ability It is very strong and widely applicable to small sample corpus, but in the classification experiment of large-scale corpus, the classification effect of support vector machine is not very good; the main advantage of the class center vector method is that the corpus is analyzed before the classification experiment. Substantial reduction, so the calculation amount of the classification experiment is small, and the classification efficiency is high, but the dimension of the vector is too large, and the vector weight is too sparse, resulting in low classification accuracy

Images