Video Stream Temporal Layering for Frame Rate Adaptation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current transmission technologies for compressed moving images are limited by the decoding capabilities of receivers, restricting high frame frequency services and increasing costs, as receivers must support low frame frequencies, making it difficult to transition to high frame frequencies without new hardware.
Innovation Solution
A transmission device that classifies image data into multiple layers, encodes each layer, and transmits a video stream with time correction information to ensure proper decoding timing, allowing receivers to support high frame rates regardless of their decoding capabilities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If services of low frame frequency are provided to match decoding capabilities of widespread receivers, then receivers can be kept cheap and widely distributed, but the ability to provide high frame frequency services is restricted
Solution Approach 1:
The video stream is segmented into multiple layers with different temporal resolutions. The encoding unit classifies image data of pictures into a plurality of layers, where each layer corresponds to a different temporal resolution. This allows receivers to selectively decode only the layers they can handle, enabling high frame frequency services to be provided while maintaining compatibility with low-capability receivers.
Solution Approach 2:
The patent introduces a new dimension of temporal scalability by encoding video at multiple temporal resolutions simultaneously. Instead of providing separate services for different frame frequencies, the system embeds multiple temporal layers within a single bitstream, allowing receivers to operate at their native frame frequency while still accessing high frame frequency content when capable.
2Adaptability or versatility
If receivers are equipped with high frame frequency decoding capabilities, then high frame frequency services can be provided, but the cost of receivers increases
Solution Approach 1:
By segmenting the video stream into multiple temporal layers, the patent allows low-cost receivers to decode only the base layer at their native frame frequency, while high-end receivers can decode additional enhancement layers for high frame frequency content. This eliminates the need for all receivers to have high frame frequency capabilities.
Solution Approach 2:
The patent changes the temporal resolution parameter of the encoded video stream by creating multiple layers with different temporal resolutions. This allows the same bitstream to be efficiently decoded at different frame frequencies, enabling high frame frequency services without requiring all receivers to support high frame rates.
3Device complexity
If temporal scalability encoding is performed with equal decoding intervals for all layers, then encoding simplicity is maintained, but decoding timing synchronization becomes problematic for receivers decoding only specific layers
Solution Approach 1:
The patent applies different decoding interval characteristics to different layers. The encoding unit performs encoding such that decoding intervals of pictures in the same layer are equal, while decoding timings of pictures across different layers are synchronized. This local differentiation ensures that receivers decoding only specific layers can maintain proper timing synchronization.
Solution Approach 2:
The patent introduces time correction information as an intermediary element that mediates between the encoding timing and decoding timing. This correction information is inserted into the bitstream to adjust decoding timings and ensure synchronization across layers, allowing receivers to properly synchronize even when decoding only specific layers.
Data Source
AI summary
It is possible to perform excellent reproduction regardless of whether or not a reception side supports a high frame rate. Image data of pictures configuring moving image data are classified into a plurality of layers, the classified image data of the pictures of each of the layers is encoded, and a video stream including the encoded image data of the pictures of each of the layers is generated. A container of a predetermined format including the generated video stream is transmitted. Time correction information for correcting a decoding timing is inserted into the encoded image data and/or a packet containing the encoded image data in association with each of the encoded image data of the pictures of a predetermined layer or a layer lower than the predetermined layer.