Scene Change Detection Using Spatial Temporal Feature Voting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing scene change detection technologies in video coding face challenges such as low robustness, high delay, high false positive rates, and slow speed due to reliance on limited features, leading to inefficient compression and quality issues in video encoding.
Innovation Solution
A method for detecting abrupt scene changes using a combination of spatial and temporal features, where feature groups are evaluated independently and through majority voting, with machine learning to deduce relationships and perform decisions, allowing for real-time operation by processing downsampled frames and reducing lookahead operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If feature extraction is performed on full resolution video, then detection accuracy is improved, but processing speed deteriorates
Solution Approach 1:
The video processing is segmented into multiple stages: first processing downsampled frames at low resolution for initial scene change detection, then selectively processing only suspicious regions at full resolution. This segmentation allows the system to maintain high detection accuracy while dramatically improving processing speed by avoiding full-resolution analysis of all frames.
Solution Approach 2:
Instead of performing full-resolution feature extraction on all video frames, the system applies partial action by processing only downsampled frames initially, and then applies excessive action (full-resolution processing) only to frames that require further verification. This selective approach optimizes the balance between accuracy and speed.
2Reliability
If multiple frames are buffered for analysis, then detection reliability is improved, but processing delay increases
Solution Approach 1:
The analysis is segmented into a two-pass approach: first pass uses minimal buffering (current and previous frame) with downsampled processing for quick detection, second pass uses selective buffering only when scene changes are detected. This reduces average buffering requirements while maintaining reliability through targeted verification.
Solution Approach 2:
The system performs preliminary scene change detection using downsampled frames before committing to full-resolution analysis. This preliminary action allows the system to identify and flag suspicious frames early, reducing the need to buffer and process all frames at full resolution, thereby reducing delay while maintaining reliability.
3Reliability
If comprehensive features are used, then detection robustness is improved, but system complexity increases
Solution Approach 1:
The feature extraction process is segmented into two levels: basic features extracted from downsampled frames for all frames, and advanced features extracted only from suspicious frames. This segmentation reduces average computational complexity while maintaining robustness through selective application of comprehensive feature analysis.
Solution Approach 2:
The system applies partial feature extraction (basic features only) to most frames, and excessive feature extraction (comprehensive features) only to frames where scene changes are suspected. This partial/excessive approach maintains detection robustness while significantly reducing overall system complexity and computational burden.
4Reliability
If full resolution processing is used, then false positive rate is reduced, but processing time increases
Solution Approach 1:
The processing pipeline is segmented into fast preliminary detection using downsampled frames and slower verification using full-resolution frames. This segmentation allows the system to quickly filter out non-scene-change frames at low resolution, then apply full-resolution analysis only to ambiguous cases, thereby reducing false positives without proportionally increasing processing time.
Solution Approach 2:
The system applies partial processing (downsampled analysis) to the majority of frames to quickly eliminate obvious non-scene-changes, and applies excessive processing (full-resolution analysis) only to frames requiring verification. This approach minimizes the time spent on full-resolution processing while maintaining low false positive rates through selective verification.
Data Source
AI summary
Methods, apparatuses and systems may provide for technology that quickly and accurately detects scene changes by evaluating a current frame based at least in part on a plurality of feature groups. Each of the feature groups may include a plurality of feature values determined from individual features. The individual features may include one or more spatial features of the current frame and one or more temporal features of the current frame as compared with previously evaluated temporal features of a previous reference frame. A determination of whether a scene change has occurred at the current frame may be made based at least in part on a majority vote among the plurality of feature groups.