Short text classification method and system

Abstract

The invention discloses a short text classification method and system. The method comprises the following steps: performing word segmentation processing on a short text, performing part-of-speech tagging on words obtained after word segmentation, and obtaining part-of-speech vectors of the words; multiplying the part-of-speech vectors by corresponding part-of-speech weight values, and obtaining new part-of-speech vectors; multiplying word vectors of the words by corresponding TF-IDF weight values to obtain weighted word vectors; splicing the weighted word vectors with the new part-of-speech vectors, and obtaining weighted spliced word vectors; overlaying the weighted spliced word vectors of the words with the weighted spliced word vectors of corresponding adjacency words to obtain adjacency word vectors; and classifying the short text according to the adjacency word vectors of the words, thus a short text classification result with relatively high accuracy is obtained; meanwhile, the method classifies the short text by calculating similarity of the short text with the candidate document according to the adjacency word vectors, thus computational efficiency is further improved.

Description

technical field [0001] The invention relates to the technical field of computer natural language processing, in particular to a short text classification method and a system for applying the method. Background technique [0002] Short text classification is a branch of shallow natural language processing, and its processing objects are various forms of short text corpus. In the field of natural language processing, how to express a word or sentence has always been a difficult problem. Although existing text classification methods such as word embedding technology (Word Embedding) can make the expression of words more and more powerful, however, the ability to express sentences or short texts still needs to be improved. [0003] At present, the better methods are word2vec, sentence2vec, tf-idf, LSI and so on. However, the above methods either ignore the semantic information and position information of the words in the sentence, or ignore the association relationship between...

AI Technical Summary

Problems solved by technology

However, the above methods either ignore the semantic information and position information of the words in the sentence, or ignore the relationship between adjacent words, so that the learned results lose part of the semantics, thus making the classification results inaccurate

Images