Weak Supervision Content Filtering for Social Media Streams
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Social media platforms generate vast amounts of data, making it challenging to filter out relevant content in real-time due to the high volume and speed of information, with most data being personal and not of interest to users seeking specific information.
Innovation Solution
A system and method for automatic topic-relevant content filtering from social media text streams using weak supervision, which involves monitoring social media streams, applying heuristic rules to identify and filter relevant content, and continuously updating classifiers to provide real-time analysis and event storytelling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual filtering methods are used to identify relevant content, then filtering accuracy can be maintained, but the processing speed and scalability deteriorate due to the high volume of social media data
Solution Approach 1:
The system performs preliminary action by pre-processing social media streams to identify potential relevant content using automated feature extraction and initial filtering mechanisms before applying more sophisticated classification. This prepares the data in advance, allowing faster processing when actual filtering is needed, thus resolving the contradiction between maintaining accuracy and improving processing speed.
Solution Approach 2:
The patent introduces intermediary components including automated feature extractors, preliminary classifiers, and hybrid filtering mechanisms that bridge between raw social media data and final filtered results. These intermediaries process data in stages, maintaining filtering accuracy through multiple checkpoints while improving overall processing speed by distributing the computational load across multiple processing layers.
2Measurement precision
If extensive manual labeling is performed to train classifiers, then classification accuracy improves, but the time and resource requirements increase significantly
Solution Approach 1:
The system applies partial labeling by training classifiers on a subset of manually labeled data combined with automatically labeled data. Rather than requiring complete manual labeling of all training data, the system uses a hybrid approach where automated labeling handles the majority of cases and manual labeling provides quality control on selected samples, thus maintaining classification accuracy while dramatically reducing labeling time and resources.
Solution Approach 2:
The patent implements self-service through automated feature extraction and automated labeling mechanisms that enable the system to generate its own training data. The classifiers automatically extract features from social media streams and generate preliminary labels, which are then used to train and retrain the system with minimal human intervention, eliminating the need for extensive manual labeling while maintaining or improving classification accuracy over time.
3Speed
If real-time processing of social media streams is implemented, then responsiveness to current events improves, but the computational complexity and resource consumption increase
Solution Approach 1:
The patent segments the social media stream processing into distinct modular components including feature extraction modules, classification modules, and filtering modules that operate independently in parallel. This segmentation allows real-time processing by distributing computational tasks across multiple independent processing units, maintaining responsiveness to current events while managing computational complexity through modular architecture and parallel processing.
Data Source
AI summary
Presented are a system, method, and apparatus for automatic topic relevant content filtering from social media text streams using weak supervision. A computing device utilizes heuristic rules allowing topic filtering and a data stream data chunk identifier. A plurality of messages are transmitted as streaming message data from a social media network in real-time. The messages are split into a plurality of data stream data chunks according to the data stream data chunk identifier. A rule-based labeled data set L0 is built from one or more data instances in the first stream data chunk. An initial classifier is built based upon features of L0. The initial classifier is applied to a next data stream data chunk to build a labeled data set L1. A subset of representative instances S1 is selected from labeled data set L1. A first representative classifier C1 is constructed from representative instance S1.


