Text classification method and system based on part-of-speech classification

Abstract

The invention discloses a text classification method based on part-of-speech classification. The method comprises the steps of acquiring a training text set and a test text set from a network; preprocessing the training text set and the test text set to obtain a plurality of word sets of each text in the training text set and the test text set; training a text topic generation model LDA by takingthe obtained word sets of each text as inputs to obtain a text-word set-topic mixed probability distribution model of each text under different topic numbers; by using an SVM-train function, carryingout classifier training on a plurality of text-word set-topic mixed probability distribution models to obtain a plurality of trained classifiers; and by using the text-word set-topic mixed probabilitydistribution models as inputs of the trained classifiers, carrying out SVM class prediction. The method can solve the technical problems of high dimension of needed characteristic words during modeltraining, low classification accuracy and poor generalization capability of the classifiers in an existing method.

Description

technical field [0001] The invention belongs to the technical field of computer deep learning, and more specifically relates to a text classification method and system based on part-of-speech classification. Background technique [0002] With the wide application of various social software and self-media software, the data generated by Internet platforms every day is also increasing rapidly. These data mainly include pictures, voice, text, etc., among which text is the main one. In order to classify these massive data, it is necessary to rely on manual screening and extraction of these massive data, which will consume a lot of time and energy, and the effect of classification is not satisfactory. Text classification technology came into being just to improve the efficiency and accuracy of text classification. [0003] The existing text classification methods mainly use the text topic generation model (Latent Dirichletallocation, referred to as LDA), which can be used to ide...

AI Technical Summary

Problems solved by technology

[0005] In view of the above defects or improvement needs of the prior art, the present invention provides a text classification method and system based on part-of-speech classification. The technical problems of low accuracy and poor generalization ability of the classifier

Images