Event detection oriented multi-strategy media data stream filtering method and device thereof

Abstract

The invention relates to an event detection oriented multi-strategy media data stream filtering method and a device thereof. The method comprises the following steps: in an offline stage, according tothe collected media user data, a garbage user database and an application source blacklist list are constructed; in the online identification stage, media data are filtered through the trash user database and the application source blacklist list, and non-event media data are filtered through the media content and context characteristics, and the media data are online clustered to identify the event clusters, and the media data in the event clusters are purified. The invention effectively solves the influence of noise data and other non-event data in the micro-blog data stream on the micro-blog event detection. The method can clean up most non-event micro-blogs in micro-blog data stream, effectively improve the performance of micro-blog event detection, real-time, practical, is convenientfor extraction of hot topics and emergencies in time, and has an important guiding significance for new media data stream processing technology.

Description

technical field [0001] The invention belongs to the technical field of media data processing, and in particular relates to an event detection-oriented multi-strategy media data flow filtering method and a device thereof. Background technique [0002] As a typical representative of emerging media, Weibo is an important platform that can conveniently and quickly publish opinions, share and disseminate information. Due to the convenience, real-time and interactive nature of Weibo, its superiority even exceeds that of traditional media and portal websites in the reporting and dissemination of many hot spots and major events of public concern. Weibo has become an important platform for intelligence collection, marketing and public opinion monitoring An important source of information for the industry. Using event detection technology for Weibo, current social hot topics and major emergencies can be extracted from massive Weibo data, so as to better help users understand news tre...

AI Technical Summary

Problems solved by technology

However, in addition to some hot news and reports of emergencies, Weibo is also filled with a lot of useless information, including advertising information, daily chores, Internet rumors, and spam information automatically generated by the server. How to combine these spam information with Differentiating meaningful event microblogs has become one of the main challenges of event detection in microblog data streams.

In response to this problem, existing scholars use some filtering strategies to purify the microblog data to improve the performance of microblog event detection, which purifies the microblog data stream to a certain extent and improves the performance of event detection, but the filtering strategies they use Relatively simple, the purification effect is limited, and the purification effect cannot be evaluated

Images