Short text characteristic expanding method based on semantic atlas

Abstract

The invention discloses a short text characteristic expanding method based on a semantic atlas. The method includes the steps: performing subject modeling by the aid of a training data set of a short text, and extracting subject term distribution; reordering the subject term distribution; building a candidate keyword dictionary and a subject-keyword semantic atlas; calculating comprehensive similarity degree evaluation of candidate keywords and seed keywords based on a link analysis method, and selecting the most similar candidate keywords to finish expanding the short text. Compared with a short text characteristic representation method based on a language model, the method is simple to operate and high in execution efficiency, and semantic correlation information between the keywords is sufficiently used. Compared with a traditional short text characteristic representation method based on a word bag model, the problems of data sparseness and semantic sensitivity are effectively relieved, and the method is independent of external large-scale auxiliary training corpus or a search engine.

Description

technical field [0001] The present invention relates to the technical field of text mining, and is a short text feature extension method based on topic-keyword semantic map and link analysis, which can be applied to feature representation in short text classification and clustering tasks, and finally applied to knowledge question answering, Subfields such as user intent understanding and intelligent retrieval. Background technique [0002] With the advent of the era of big data, the Internet and various mobile terminals have generated a large amount of short text information, such as web page retrieval fragments, Weibo, product reviews, news headlines, and various micro-information, etc. Information is also being overwhelmed by the vast amount of resources. How to make the system intelligently manage and better use these massive data resources is facing a huge challenge. Therefore, a high-precision short text classification method can help the system to deepen the understa...

AI Technical Summary

Problems solved by technology

[0006] Aiming at the above two main problems, the present invention proposes a short text feature extension method based on semantic maps, which solves the problem of data sparsity and semantic sensitivity in the representation of short text features by the traditional bag-of-words model, and finally improves short text features. Ben's classification performance

Images