Motion Compensated Residual Score for Scene Change Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Scene transitions such as cuts, fades, and dissolves complicate video encoding efficiency due to increased temporal entropy, making it difficult for encoders to predictably allocate bits, and existing methods for detecting scene changes are fragile and inaccurate.
Innovation Solution
The implementation of a Motion Compensated Temporal Filter (MCTF) for pre-processing video frames to generate a Motion Compensated Residual (MCR) score, which is used to accurately detect scene changes by comparing sliding means and medians, allowing for more efficient bit allocation and determination of I-picture locations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional luma statistics methods are used for scene change detection, then the detection process is simple, but the accuracy is low and the method is fragile due to similar luma variations in different scenes
Solution Approach 1:
The patent changes the detection parameter from luma statistics to motion compensated residual (MCR) statistics. By transforming the residual signal through motion compensation and then applying statistical tests on the compensated residuals, the system achieves more accurate scene change detection that is not affected by similar luma variations in different scenes.
Solution Approach 2:
The patent introduces motion compensated residual (MCR) as an intermediary representation between the original video frames and the scene change detection process. The MCR captures motion information while removing temporal redundancy, providing a more robust feature for detecting scene changes compared to direct luma analysis.
2Productivity
If scene transitions are not accurately detected, then encoding can proceed without interruption, but bit allocation becomes inefficient due to increased temporal entropy
Solution Approach 1:
The patent implements a feedback mechanism where scene change detection results are fed back to the encoder to adjust bit allocation and reference picture management. When scene changes are detected using MCR statistics, the encoder can dynamically adjust its parameters to maintain efficient encoding despite the increased temporal entropy introduced by transitions.
Data Source
AI summary
A method is provided to better detect a scene change to provide a prediction to an encoder to enable more efficient encoding. The method uses a Motion Compensated Temporal Filter (MCTF) that provides motion estimation and is located prior to an encoder. The MCTF provides a Motion Compensated Residual (MCR) used to detect the scene change transition. When a scene is relatively stable, the MCR score is also relatively stable. However, when a scene transition is in process, the MCR score behavior changes, Algorithmically, the MCR score is used by comparing the sliding mean of the MCR score to the sliding median. This comparison highlights the transition points. In the case of a scene cut, the MCR score exhibits a distinct spike. In the case of a fade or dissolve, the MCR score exhibits a transitional period of degradation followed by recovery. By implementing the above detection using the MCR, the location of the I-pictures in the downstream encoding process can be accurately determined for the encoder.