Structured Event Object Creation from Sparse Text Streams
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems fail to effectively detect and process sparsely reported events from Twitter, which are critical for enterprises due to their local importance and limited reporting, and struggle with correlating unstructured short text messages into structured event objects for real-time analysis.
Innovation Solution
An automated system and method that filters and classifies tweets using regular expressions and supervised classification, followed by textual similarity scoring and information extraction to create structured event objects, employing Locality Sensitive Hashing (LSH) for efficient correlation and semantic analysis to generate unique event objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If keyword trend observation is used for event detection, then events with many tweets can be detected, but sparsely reported events cannot be detected
Solution Approach 1:
The patent segments the event detection process into multiple independent modules: a filtering module that identifies potential event tweets using regular expressions, a classification module that categorizes tweets using supervised learning, and a correlation module that links related tweets. This segmentation allows the system to handle sparsely reported events by processing each tweet individually rather than relying on volume-based keyword trends.
Solution Approach 2:
The patent performs preliminary filtering and classification of tweets before correlation. By pre-processing tweets to identify and categorize potential event reports in advance, the system can efficiently detect sparsely reported events when they occur, rather than waiting for volume accumulation.
2Reliability
If all tweets are compared for correlation, then comprehensive event detection is achieved, but processing time and computational resources increase significantly
Solution Approach 1:
The patent applies preliminary filtering using regular expressions and supervised classification to identify potential event tweets before correlation. This pre-processing step reduces the tweet pool from tens of messages per second to only relevant candidates, making subsequent correlation computationally feasible while maintaining comprehensive event detection.
Solution Approach 2:
The correlation process is segmented into stages: first filtering tweets by relevance, then classifying them by event type, and finally correlating only within classified groups. This segmentation dramatically reduces the comparison scope from all tweets to only relevant subsets, improving processing speed while preserving correlation accuracy.
3Ease of manufacture
If clustering based on word similarity is used, then tweets describing the same aspect are grouped, but tweets describing different aspects of the same event are not correlated
Solution Approach 1:
The patent performs preliminary classification of tweets into event types and aspects using supervised learning before correlation. By pre-organizing tweets according to their semantic content and event characteristics, the system enables comprehensive correlation across different aspects of the same event, rather than only grouping tweets with identical wording.
Data Source
AI summary
Disclosed is a system and method for converting a voluminous stream of unstructured short text messages into event-objects of specific event type that may be of potential interest to users at far away locations. The method of structuring involves detecting long tail of events in spite of their sparsity. This is followed by extracting and correlating detected short text messages that describe same event type to create structured event objects.