Video Frame Extraction for Variable Speed Playback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video playback technologies face challenges in meeting high resolution and frame rate requirements, especially during fast playback at variable speeds, due to limitations in CPU and SoC hardware decoding capabilities, leading to increased power consumption and reduced device endurance.
Innovation Solution
The method involves determining frame dropping positions based on presentation time stamps (PTS) and non-reference frame flags, allowing for level-by-level frame extraction and dropping, which reduces computational load and power consumption while maintaining a high playback frame rate.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If CPU software decoding or SoC hardware decoding is used for high resolution and high frame rate video playback, then video quality is improved, but power consumption increases and device endurance decreases
Solution Approach 1:
The patent extracts and removes redundant or low-priority frames from the video sequence based on PTS analysis and frame type identification. By extracting only essential frames for decoding and display, the system reduces the computational load on CPU/SoC while maintaining perceived video quality, thus lowering power consumption during fast playback.
Solution Approach 2:
Instead of decoding and processing every frame in the video sequence, the patent applies partial action by selectively decoding only the necessary frames identified through PTS-based frame extraction. This partial processing approach maintains adequate video quality for fast playback while significantly reducing power consumption compared to full frame processing.
2Speed
If frame rate is increased for fast playback, then playback speed is improved, but computational load increases and device endurance decreases
Solution Approach 1:
The patent extracts frames at optimized intervals based on PTS values and frame type information, rather than processing all frames at the maximum frame rate. This selective extraction reduces the number of frames requiring decoding and rendering, thereby lowering computational load while maintaining the desired fast playback speed.
Solution Approach 2:
The system applies partial action by processing only a subset of frames necessary to achieve the target playback speed. By identifying and processing only critical frames (e.g., keyframes and selected intermediate frames) rather than all frames, the computational load is reduced while still delivering smooth fast playback experience.
3Manufacturing precision
If all frames are processed for high quality playback, then video quality is maintained, but processing time increases and efficiency decreases
Solution Approach 1:
The patent extracts and identifies frame types (reference frames, non-reference frames) using PTS information before decoding. By extracting only the essential frames that contribute significantly to perceived video quality and removing redundant frames, the system improves processing efficiency without substantially degrading video quality during fast playback.
4Use of energy by moving object
If frame dropping rate is increased for fast playback, then power consumption is reduced, but video quality may deteriorate
Solution Approach 1:
The patent applies local quality by differentiating between reference frames and non-reference frames, and selectively maintaining reference frames while dropping non-reference frames. This localized quality preservation strategy ensures that frames critical for video reconstruction (reference frames) are retained, maintaining acceptable video quality even with high frame dropping rates during fast playback.
Solution Approach 2:
The system applies partial action by selectively preserving only the necessary portion of frames (reference frames) while dropping others (non-reference frames). This partial frame retention approach reduces power consumption through high frame dropping rates while maintaining sufficient video quality by keeping the essential reference frames needed for decoding and display.
Data Source
AI summary
A method for frame extraction processing of a video includes obtaining an encoded image of a video sequence; obtaining presentation time stamps (PTSes) and non-reference frame flags of the encoded image, wherein the encoded image has an encoding structure with time domain levels; determining frame dropping positions of the video sequence based on the time domain levels, the PTSes, and the non-reference frame flags; and performing a frame extraction operation on the video sequence based on the frame dropping positions.