Motion-Constrained Tile Sets SEI Message Extension for Interactive Video Streaming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The motion-constrained tile sets in video coding do not have the functionality to signal all tiles as independently decodable and to indicate Regions of Interest (ROIs) that may have more than one tile per ROI, limiting interactivity and flexibility in applications like UHDTV and interactive streaming.
Innovation Solution
An extension to the motion-constrained tile sets SEI message is introduced, enabling all tiles to be independently decodable and allowing multiple tiles per ROI, with the ability to redefine ROIs based on user or director interactivity, and supporting applications like UHDTV, dynamic zoom-in, and smart surveillance by signaling which tiles are outside the selected ROI to reduce bitrate.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the motion-constrained tile sets SEI message is used in the original form, then the video coding structure is maintained, but the functionality to signal all tiles as independently decodable and to indicate ROIs with multiple tiles is lost
Solution Approach 1:
The patent segments the SEI message into distinct components: a flag indicating whether all tiles are independently decodable, and separate ROI structure definitions that can contain multiple tiles each. This segmentation allows the message to convey complex information about tile independence and ROI compositions without creating an undecipherable monolithic structure.
Solution Approach 2:
The patent adds a new dimensional layer to the SEI message by introducing a hierarchical structure where ROIs can contain multiple tiles. This transforms the traditional one-to-one mapping between ROIs and tiles into a many-to-one relationship, enabling more versatile ROI definitions while maintaining the underlying message structure.
2Adaptability or versatility
If all tiles are made independently decodable, then interactivity and flexibility are improved, but the bitrate increases due to redundant encoding information
Solution Approach 1:
The patent applies partial action by allowing selective independent decodability through a flag that can be set for specific tiles or regions. Instead of forcing all tiles to be independently decodable (excessive action), the system enables this capability only where needed, thereby maintaining interactivity and flexibility while avoiding unnecessary bitrate increase from redundant encoding throughout the entire video.
3Adaptability or versatility
If multiple tiles per ROI are allowed, then the ability to define flexible regions is improved, but the complexity of mapping and decoding increases
Solution Approach 1:
The patent segments the ROI definition into discrete tile references within an ROI structure. Each ROI can contain a list of tile indices, creating a clear segmented mapping between ROIs and tiles. This segmentation simplifies the decoding process by providing explicit tile-to-ROI relationships rather than requiring complex geometric calculations.
Solution Approach 2:
The patent uses copying by referencing tile indices rather than duplicating full tile data for each ROI. The tile information is copied or referenced multiple times across different ROI structures, allowing flexible ROI definitions while avoiding redundant storage and processing of actual tile content.
4Quantity of substance
If tiles outside ROI are encoded as gray tiles, then bitrate is reduced, but the visual quality and information completeness are compromised
Solution Approach 1:
The patent extracts only the necessary visual information by encoding gray tiles for regions outside the selected ROI. This extraction principle allows the system to remove unnecessary detailed encoding from non-critical regions while preserving full quality in the selected ROI, thereby reducing overall bitrate without significantly impacting the perceived visual information for the user's area of interest.
Data Source
AI summary
An extension to the motion-constrained tile sets SEI message provides functionality to signal all tiles are independently decodable and to signal the ROIs that may have more than one tile per ROI. With this extension, the functionality to redefine any independently decodable region-of-interest in a CVS at a coding tree unit level based on user interactivity is enabled. The extension supports the interactivity utilized in various applications such as interactive Ultra High Definition Television (UHDTV), dynamic high-quality zoom-in application, interactive on-demand, e-learning, smart surveillance and many other applications. Additionally, the temporal MCTS SEI message is able to be used by an encoder for tiled streaming to signal explicitly to the decoder that the decoder need only to display the ROI.