Hot topic detection method of Chinese micro-blogs

Abstract

The invention relates to a hot topic detection method of Chinese micro-blogs. The hot topic detection method includes the following steps of (1) filtering junk micro-blogs based on a certain junk filtering rule at first, (2) preliminarily gathering keywords distributed in the micro-blogs to obtain a primary word set expressing topics, retrieving the most similar first (i)k( / i) micro-blogs of each micro-blog, and then enriching the characteristics of each micro-blog by virtue of the result of retrieval and the result of preliminary keyword gathering to obtain the enriched characteristic vector of the characteristics of each micro-blog, and (3) clustering all of the micro-blogs by an incremental clustering method based on the enriched characteristic vector of the characteristics of each micro-blog to obtain a clustered topic set, next, performing topic popularity calculation on the clustered topic set through a certain topic popularity calculation formula, and finally, obtaining a hot topic list. The method is capable of performing hot topic detection on the Chinese micro-blogs efficiently and accurately, and also high in detection speed, high in accuracy, wide in application range and high in applicability.

Description

Technical field [0001] The present invention relates to the technical field of topic detection and tracking, and more specifically, to a hot topic detection method for Chinese microblogs, which can be applied to hot topic detection and popularity ranking, and is suitable for Chinese microblogs, including Sina Weibo and Tencent Weibo. Blog, Netease Weibo, etc. Background technique [0002] The Topic Detection and Tracking (TDT) task started in 1996. A topic contains a series of events or activities, or accompanying directly related events and activities. A TDT event represents what happened at a specific time and occasion, together with all necessary preconditions and unavoidable consequences. [0003] After more than ten years of rapid development in topic detection and tracking, a series of mature theories have been developed, including hidden Markov models, aging theory, time series analysis, LDA, etc. [0004] Hot topics are topics that frequently appear in a period of time. A ...

AI Technical Summary

Problems solved by technology

[0010] In the existing technology, starting from the modeling point of view, for example, by improving LDA to apply to Weibo's Author-topic and Twitter-LDA models, etc., it can be effectively used for Weibo topic modeling, but the complexity of this type of algorithm Generally high, it is not conducive to detecting hot topics from large-scale Weibo data collections

[0011] However, most of the existing Chinese microblog hot topic detection methods do not fully consider the following three issues: 1) there is a large amount of noise information in microblogs; 2) short texts in microblogs cause information loss; 3) massive microblog data belt coming performance challenges

Images