Real-Time Streaming Video Highlight Detection Using Cascade Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems are unable to automatically detect and compile highlights from streaming video in real-time, requiring human editors and only generating highlights after a game has ended, thus lacking an online mechanism for on-the-fly segmentation of streaming media.
Innovation Solution
A novel cascade prediction model, referred to as a scene-highlight classifier, analyzes streaming video frames to identify game scenes and score them for highlights, enabling automatic real-time identification and creation of video clips based on predetermined thresholds.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional systems are used to generate highlights, then highlights can be produced with human editors, but the process cannot occur in real-time and only after a game has ended
Solution Approach 1:
The patent replaces the mechanical system of human editors with an automated computerized system that uses machine learning models (scene classifier and highlight classifier) to detect and generate highlights automatically, enabling real-time processing without human intervention
Solution Approach 2:
The system performs preliminary classification of video frames into scene types (e.g., gameplay, commentary, advertisements) before highlight detection, allowing the highlight classifier to focus only on relevant frames and achieve real-time performance
2Productivity
If automated real-time highlight detection is implemented, then immediate highlights can be provided, but the system complexity increases significantly
Solution Approach 1:
The patent divides the complex highlight detection task into two separate specialized classifiers: a scene classifier that categorizes video frames into different scene types, and a highlight classifier that detects highlights within identified gameplay scenes. This segmentation reduces the complexity of each individual classifier while maintaining overall system effectiveness
Solution Approach 2:
The scene classifier acts as an intermediary between the raw video input and the highlight classifier, filtering and categorizing frames before they reach the highlight detection stage. This intermediary layer simplifies the workload of the highlight classifier and enables real-time processing
3Reliability
If manual highlight editing is used, then high quality highlights can be produced, but human intervention is required and processing is slow
Solution Approach 1:
The system enables self-service automated highlight generation where the computerized system independently detects, classifies, and generates highlights without requiring human editors, while maintaining reliable quality through trained machine learning models that identify gameplay patterns and highlight moments
Data Source
AI summary
Disclosed are systems and methods for improving interactions with and between computers in content generating, searching, hosting and/or providing systems supported by or configured with personal computing devices, servers and/or platforms. The systems interact to identify and retrieve data within or across platforms, which can be used to improve the quality of data used in processing interactions between or among processors in such systems. The disclosed systems and methods provide systems and methods for automatically detecting and rendering highlights from streaming videos in real-time. As a streaming video is being broadcast over the Internet, the disclosed systems and methods determine each type of scene from the streaming video, and automatically score highlight scenes. The scored highlight scenes are then communicated to users as compiled video segments, which can be over any type of channel or platform accessible to a user's device and network that enables content rendering and user interaction.