A Short Text Topic Mining Method Based on Semantic Word Network

Abstract

The invention discloses a short text topic mining method based on a semantic word network, comprising the following steps: 1) model initialization stage: external corpus collection in related fields, corpus preprocessing, parameter setting, etc.; 2) topic unit construction stage: constructing semantic words 3) Model training stage: use the Gibbs sampling method to sample model variables, and judge whether the model has reached the convergence condition; 4) Result output stage: according to the model After the training, the sampling results of each variable are obtained to obtain the topic distribution of each word triangle, and then the topic distribution of the original document is calculated. The present invention combines the semantic information learned from the external corpus with the word triangular topic structure and applies it to short text topic mining. Compared with the traditional word pair topic model, this method provides a method that integrates external prior knowledge into the traditional topic model. solution, and the quality of mining topics has improved significantly.

Description

technical field [0001] The invention relates to a short text topic mining method, in particular to a short text topic mining method based on a semantic word network, which solves the problem of low topic quality in the case of sparse short text features in common topic mining methods. Background technique [0002] With the continuous acceleration of the pace of social development and the "short, smooth and fast" user experience brought by smart mobile terminals, people's communication on the Internet is becoming more and more fragmented. Therefore, short text data occupies an increasingly important position in today's network information interaction, such as social network status, Weibo text messages, traditional news titles, short video titles, and question-and-answer websites, etc., all appear in the form of short text. And with the rise of large-scale companies such as Weibo, Zhihu, Facebook, and Twitter, short text data is also generated and accumulated at a great speed....

AI Technical Summary

Problems solved by technology

[0005] Purpose of the invention: The technical problem to be solved by the present invention is that when the traditional topic model considers word co-occurrence information in response to the scarcity of short text data features, the quality of mining topics is not high enough due to the introduced noise information and ignored semantic information

Images