Method and device for classifying texts, storage medium and processor

Abstract

The embodiment of the invention provides a method and a device for classifying texts, a storage medium and a processor, and belongs to the technical field of computers. The method comprises the stepsof performing word segmentation on a to-be-classified text; determining a word vector corresponding to each word obtained by performing word segmentation on the to-be-classified text based on a word vector model, and forming a matrix by the word vectors corresponding to the words belonging to one sentence; processing each matrix based on the sentence vector model to obtain a sentence vector corresponding to each matrix; and processing each sentence vector based on the sentence classification model to obtain a category score vector corresponding to each sentence vector, and determining the typeof the sentence vector corresponding to the category score vector according to each category score vector to realize classification of the to-be-classified text. Therefore, the defects that the constructed word frequency or feature matrix is very sparse and the relationship between words is ignored when the short texts are classified are overcome, and the effect of classifying the texts is improved.

Description

technical field [0001] The present invention relates to the field of computer technology, in particular to a method and device, a storage medium and a processor for classifying text. Background technique [0002] Text classification is a basic task in natural language processing, including sentence-level and chapter-level text classification, namely short text classification and long text classification. Text classification is widely used, and common application scenarios include spam classification, sentiment analysis, and news topic classification. For short text classification, the main application is automatic question answering system and query classification in search engines. Traditional text classification methods first perform text preprocessing, then feature extraction, and then classifier selection and training. Text preprocessing is usually to segment the text, remove stop words, part-of-speech tagging, etc. Conventional features usually use TF-IDF, and also i...

AI Technical Summary

Problems solved by technology

However, since query questions are generally short and irregular, there are very few words that can be provided after preprocessing such as word segmentation and stop word removal, and the constructed feature matrix is ​​very sparse, coupled with the lack of information such as semantics and word order, which leads to classification The effect is not ideal

Images