Video Encoding Temporal Scalability for Frame Rate Adaptation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video encoding and playback technologies struggle to efficiently adapt video frame rates to match the capabilities of different playback devices, leading to inefficient resource usage and suboptimal viewing experiences.
Innovation Solution
The system encodes video into a base layer and enhancement layers, using temporal identifiers to determine layer association and retain order, allowing devices to request and stream only necessary layers based on their capabilities, merging them for playback at different frame rates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If video is encoded using a single fixed frame rate, then the encoding process is simple, but the playback device cannot adapt to different frame rate requirements of various devices
Solution Approach 1:
The video stream is segmented into multiple temporal layers (base layer and enhancement layers) with different frame rates. The base layer contains frames at a lower frame rate while the enhancement layer contains additional frames at a higher frame rate. This segmentation allows playback devices to select and decode only the layers they can handle, achieving frame rate adaptability without requiring the encoder to handle every possible device configuration.
Solution Approach 2:
The patent introduces a temporal dimension to the encoding structure by creating multiple temporal layers with different frame rates. Instead of encoding a single frame rate and trying to adapt to different devices through post-processing, the system encodes video in multiple temporal dimensions simultaneously, allowing devices to select the appropriate temporal layer for their capabilities.
2Reliability
If all video layers are transmitted to ensure highest quality playback, then playback quality is maximized, but network resources are wasted on devices that cannot handle high frame rates
Solution Approach 1:
The system dynamically adapts the transmitted video layers based on the capabilities of the receiving device. Playback devices signal their frame rate capability to the server, which then dynamically selects which temporal layers to transmit. This dynamic adaptation ensures that high-quality video is transmitted only when necessary, optimizing network resource consumption while maintaining playback quality for capable devices.
Solution Approach 2:
The system changes the parameter of frame rate by providing multiple temporal layers with different frame rates (e.g., 30fps base layer and 60fps enhancement layer). This allows the system to adjust the effective frame rate parameter to match device capabilities, ensuring reliable playback quality for devices that support higher frame rates while avoiding unnecessary transmission of high-frame-rate video to devices that can only handle lower frame rates.
3Productivity
If video is encoded at high frame rate for smooth playback, then viewing experience is improved, but the data size increases and transmission time is extended
Solution Approach 1:
The system applies partial action by transmitting only the necessary temporal layers based on device capability. Instead of always transmitting the full high-frame-rate video (excessive action), the system transmits only the base layer to devices that cannot handle enhancement layers, and only transmits enhancement layers when needed. This partial transmission approach reduces transmission time and data size while maintaining playback smoothness for devices that can handle higher frame rates.
Data Source
AI summary
Systems and methods in accordance with embodiments of this invention provide for encoding and playing back video at different frame rates using enhancement layers. In a number of embodiments, video is encoded using temporal scalability involving the creation of a base layer at a first frame rate and an enhancement layer including additional frames enabling playback at a second higher frame rate. The second higher frame rate can also be referred to as an enhanced frame rate. In a number of embodiments, the base and enhancement layers are stored in one or more container files that contain metadata describing the enhancement layer. Based on the capabilities of a playback device, it can select the particular frame rate at which to playback encoded video.