Violative Content Prediction Using Audience Overlap Signals
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing media content detection systems are limited in identifying violative content beyond keyword analysis, particularly for content that violates policies such as violence, misinformation, and illegal content, especially in large and evolving media libraries.
Innovation Solution
A system that utilizes user-item interaction data to train a machine-learning model, analyzing consumption histories of media items to predict potential violative content by comparing audience overlap with known violative content, assigning scores, and flagging items for human review.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If keyword-based detection methods are used to identify violative content, then the detection process is simple and fast, but the precision and coverage of violative content identification are limited
Solution Approach 1:
The patent introduces user interaction data as an intermediary signal to bridge the gap between content and violative classification. Instead of directly analyzing content features, the system uses playback history and user behavior patterns as a mediator to infer violative nature, thereby improving precision without requiring complex content analysis of subtle violative elements
Solution Approach 2:
The patent replaces traditional mechanical keyword-matching systems with a machine learning model that processes user interaction data. This substitution transitions from rule-based mechanical detection to intelligent pattern recognition, significantly improving detection precision for diverse violative content while the automated nature keeps operational complexity manageable
2Measurement precision
If traditional content analysis methods are used to detect violative content in large media libraries, then the system structure is simple, but the ability to identify diverse violative content types with high precision is insufficient
Solution Approach 1:
The patent creates a universal detection framework that handles multiple violative content types (violence, misinformation, illegal content, etc.) through a single machine learning model. The model processes different content types and violative patterns uniformly by analyzing user interaction patterns, achieving both high precision and broad adaptability across diverse media content
Solution Approach 2:
The patent changes the detection parameters from content-based features (keywords, transcripts) to user-behavior-based features (playback history, interaction patterns). This parameter transformation enables the system to adapt to various content types and violative patterns without requiring content-specific rules, thereby improving both precision and versatility simultaneously
3Measurement precision
If user-item interaction data is used to train a machine-learning model for predicting violative content, then the precision of identifying violative content is enhanced, but the complexity of the detection system increases
Solution Approach 1:
The patent implements a self-service detection system where the machine learning model automatically learns from user interaction data and continuously improves its detection capabilities. The system serves itself by using its own operational data (playback histories) as training material, reducing the need for external manual annotation and simplifying the overall system architecture despite the sophisticated model
4Productivity
If keyword and phrase-based detection is used for violative content, then the implementation is straightforward, but false positives occur and detection coverage is limited
Solution Approach 1:
The patent incorporates feedback loops where user interaction data continuously refines the machine learning model's predictions. The system learns from actual user behavior patterns associated with violative content, creating a feedback mechanism that reduces false positives over time while maintaining high screening efficiency. The model adapts based on confirmed violative cases and adjusts its prediction thresholds accordingly
Data Source
AI summary
An electronic device identifies a set of seed content items that correspond to violative content items. The electronic device determines, using playback histories indicating consumption of respective content items, connections between a respective content item and a first audience that has consumed the respective content item and that has consumed at least a threshold number of seed content items from the set of seed content items. The electronic device provides information corresponding to the connections as an input to a machine learning model. The electronic device receives, as an output from the machine learning model, likelihoods that respective content items are violative content items and stores a set of content items, selected using the output from the machine learning model, as candidate content items in accordance with a determination that the content item satisfies likelihood criteria.