Video Encoding Droppable Frames for Limited Decoder Resources
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Devices with limited resources struggle to decode and display video data at high frame rates, as they often lack the necessary processing power to handle the high frame rates captured by advanced cameras, leading to inefficiencies in video playback.
Innovation Solution
Incorporating a method to encode video data with 'droppable frames' that can be omitted during decoding, allowing devices to reduce the frame rate to a manageable level by dropping specific frames that do not rely on other frames for prediction, thereby enabling efficient playback on devices with limited resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If video data is captured and encoded at high frame rates, then video quality and motion smoothness are improved, but device resources are overwhelmed and real-time display becomes infeasible
Solution Approach 1:
The video stream is segmented into reference frames and non-reference frames. Reference frames are encoded at lower resolution or with lower frequency, while non-reference frames can be dropped without affecting decoding of subsequent frames. This segmentation allows the system to reduce processing load by selectively decoding only essential frames while maintaining acceptable video quality.
Solution Approach 2:
Non-essential frame data is extracted and removed from the decoding process. The patent identifies and extracts droppable frames that do not serve as reference for other frames, allowing these frames to be discarded without impacting the ability to decode and display the video stream on resource-constrained devices.
2Reliability
If all captured frames are decoded and displayed, then video quality is maintained, but device resources are exceeded and playback fails
Solution Approach 1:
The decoding process dynamically adapts to available device resources. The system determines which frames to decode based on real-time resource availability and the frame's importance (whether it serves as a reference frame). This dynamic approach ensures playback reliability across devices with varying resource constraints while maintaining video coherence.
Solution Approach 2:
The patent changes the decoding parameters by selectively applying full decoding only to reference frames while skipping or simplifying decoding of non-reference frames. This parameter change in the decoding process reduces computational demand while preserving the essential video information needed for reliable playback.
3Productivity
If frame rate is reduced to match device capabilities, then real-time display becomes feasible, but video smoothness and quality deteriorate
Solution Approach 1:
By segmenting frames into reference and non-reference categories, the system can process reference frames at the reduced device-capable frame rate while maintaining higher quality for these key frames. Non-reference frames are either dropped or processed with reduced effort, enabling overall system throughput to match device capabilities without severely compromising perceived video quality.
Data Source
AI summary
Video data with high frame rates may be displayed on devices with limited resources (e.g., decoder and/or display resources). These devices may have their resources devoted to other tasks or may not be capable to display the video data at the high frame rates. The coding method may include coding the frames such that additional droppable frames are included in the encoded video data. The decoding method may include dropping droppable frames before the encoded video data is decoded to reduce the number of frames that will be decoded and displayed. These methods may be applied to video data that has a variable frame rate and may be combined with processing the image sequence for slow motion playback.