Detection system for identifying microblog data stream of sudden event in real time

Abstract

The invention discloses a detection system for identifying the microblog data stream of a sudden event in real time, and the sudden event can be rapidly and accurately detected and identified withoutany priori knowledge about the event. According to the technical scheme, the system is characterized in that a crawler tool is used for crawling text data in real time; an entity extraction module extracts multiple types of named entities, and a trend recognition module is adopted to obtain a hot word list about different regions; an entity filtering module is used for filtering entities with no heat; a similarity calculation module is used for establishing a co-occurrence matrix in a window, calculating entity similarity and constructing an entity relation graph; a similarity filtering moduleis used for filtering edges with smaller numerical values in the entity relation graph; an entity clustering module uses a community discovery algorithm to obtain a corresponding clustering set for the entity relation graph; a clustering link module is used for continuously tracking the events in the event window; a clustering and grading module grades the clustering result subjected to the clustering link according to the number of hot words contained in the clustering result; and a data storage module stores clustering and grading information.

Description

technical field [0001] The invention belongs to the technical field of emergency event detection and identification, in particular to a detection system for real-time identification of emergency event microblog data streams. Background technique [0002] With the rapid development of Internet technology, social network services, news, forums, Weibo, and social platforms based on smart phone applications, some emerging Internet services have become important platforms for people to spread and obtain information. Especially in recent years, the development of microblogging is suddenly emerging, and it is loved by the majority of users by virtue of its real-time and convenience. People can publish and obtain relevant information about a sudden event in the "real world" at the first time. For example, the Sina Weibo account officially certified by the China National Health and Medical Commission has become the primary way for many Chinese to understand the real-time situation o...

AI Technical Summary

Problems solved by technology

At present, although there are many research results and some effective solutions in real-time event detection and recognition, most of these emergency event recognition methods only realize the detection and recognition of global events or regional events (such as countries) ( Such as large-scale natural disasters, armed conflicts, etc.), did not detect and identify small-scale events (such as local epidemics, forest fires, etc.)

In addition, some methods need to manually set the number of events, event types and other information, which often requires a priori knowledge of large materials and manual labeling data

Images