Knowledge-integrated subject model-based microblog topic detection method

Abstract

The invention discloses a knowledge-integrated subject model-based microblog topic detection method and relates to the field of natural language processing. The method comprises the following steps of firstly, acquiring microblog data and carrying out scoring, Chinese segmentation and stop word filtration on a microblog text; secondly, modeling the microblog text by using a knowledge-integrated subject model; and finally, detecting the topic by using hybrid clustering of a K-center and hierarchical clustering. By using a knowledge-integrated subject modeling method, the problem of data sparseness of a short microblog text set is effectively solved, a subject feature vector can be accurately defined and the clustering accuracy is effectively improved.

Description

technical field [0001] The invention relates to a microblog topic discovery method, in particular to a microblog topic discovery method based on a topic model of fusion knowledge. Background technique [0002] With the rapid development of the mobile Internet, more and more people use Weibo to express their opinions and opinions, and they can keep abreast of news happening around the world on Weibo. Due to the rapid increase in the number of microblog texts and the complexity of information, it is difficult for users to browse all microblog information. Therefore, obtaining hot topics has important research significance: on the one hand, it can help users quickly understand the hot spots of concern in various fields of society, and on the other hand, it can provide public opinion guidance for the field of public opinion monitoring. [0003] In recent years, the methods of microblog opinion discovery are mainly divided into two types: text clustering algorithms and topic mod...

AI Technical Summary

Problems solved by technology

However, when only traditional clustering methods are used for microblog topic discovery, the problem of high dimensionality and sparsity of feature vectors will occur, making the clustering results inaccurate

In recent years, there have been many studies on the combination of LDA-based topic models and clustering algorithms. However, the modeling results of the standard topic model LDA often produce many internally inconsistent topics.

Images