Short text topic mining method based on semantic word network

Abstract

The invention discloses a short text topic mining method based on a semantic word network. The short text topic mining method comprises the following steps: 1) a model initialization stage: collectingexternal corpora in related fields, preprocessing the corpora, setting parameters and the like; 2) a theme unit construction stage: constructing a semantic word network, searching a specific word triangular structure, calculating model prior parameters and the like; 3) a model training stage: sampling model variables by using a Gibbs sampling method, and judging whether the model reaches a convergence condition or not; and 4) a result output stage: obtaining topic distribution of each word triangle according to a sampling result of each variable after model training is finished, and calculating topic distribution of the original document. According to the method, semantic information learned by an external corpus is combined with the word triangular theme structure, the method is appliedto short text theme mining, compared with a traditional word pair theme model, the method provides a solution for integrating external priori knowledge into the traditional theme model, and the quality of the mined theme is remarkably improved.

Description

technical field [0001] The invention relates to a short text topic mining method, in particular to a short text topic mining method based on a semantic word network, which solves the problem of low topic quality in the case of sparse short text features in common topic mining methods. Background technique [0002] With the continuous acceleration of the pace of social development and the "short, smooth and fast" user experience brought by smart mobile terminals, people's communication on the Internet is becoming more and more fragmented. Therefore, short text data occupies an increasingly important position in today's network information interaction, such as social network status, Weibo text messages, traditional news titles, short video titles, and question-and-answer websites, etc., all appear in the form of short text. And with the rise of large-scale companies such as Weibo, Zhihu, Facebook, and Twitter, short text data is also generated and accumulated at a great speed....

AI Technical Summary

Problems solved by technology

[0005] Purpose of the invention: The technical problem to be solved by the present invention is that when the traditional topic model considers word co-occurrence information in response to the scarcity of short text data features, the quality of mining topics is not high enough due to the introduced noise information and ignored semantic information

Images