Text classification method and terminal device

Abstract

The invention relates to the field of information processing, and discloses a text classification method and a terminal device. According to the embodiment, the method comprises the steps of calculating a word vector matrix containing all word vectors after training samples of N text types are segmented; calculating out the feature vector of the training sample of each text type based on the word vector matrix; calculating the input amount of a back-propagating neural network according to the calculated feature vectors of the training samples; then determining a text classifier according to the back-propagating neural network; finally determining the type of a text to be tested according to the feature vector of the text to be tested and the text classifier. In CBOW pair word vectorization, the relation of a current word with a few front words and a few rear words is taken into consideration, the whole network training also has a semantic feature by being combined with the classical back-propagating neural network, the whole network can further recognize and understand the text content, and the better training effect is achieved.

Description

technical field [0001] The invention relates to the field of information processing, in particular to a text classification method and terminal equipment. Background technique [0002] Text classification refers to taking a set of texts classified by experts in advance as a training sample set, analyzing the training sample set to obtain a classification pattern, and using the derived classification pattern to classify other texts. It is mainly used in information retrieval, machine translation, automatic summarization and information filtering. [0003] In the process of realizing the present invention, the inventors of the present application found that the CBOW training method based on the hierarchical classifier Hierarchical Softmax is more beneficial to rare words, and can achieve a faster classification of this type of text. The CBOW based on the negative sampling algorithm The training method is beneficial to the text classification of common words and low-dimensiona...

AI Technical Summary

Problems solved by technology

[0003] In the process of realizing the present invention, the inventors of the present application found that the CBOW training method based on the hierarchical classifier Hierarchical Softmax is more beneficial to rare words, and can achieve a faster classification of this type of text. The CBOW based on the negative sampling algorithm The training method is beneficial to the text classification of common words and low-dimensional vectors. At the same time, when CBOW based on two different algorithms is trained, the window size usually selected is about 5. The word vectors obtained based on the above two text training methods have Certain semantic features, but cannot deeply recognize and understand the content of the text

Images