Scalable Tile Media Encapsulation for ROI Streaming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for HTTP streaming of video fail to efficiently access and combine user-selected regions of interest (ROIs) in compressed video streams, leading to data duplication and increased decoding requirements due to the lack of efficient encapsulation of spatial tiles in existing file formats like ISO BMFF.
Innovation Solution
A method for encapsulating partitioned timed media data in media segment files, allowing for independent selection and combination of scalable tiles, using mapping metadata to provide information about subsamples and their layers, reducing indexation overhead and improving parsing efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If video streams are encapsulated in traditional file formats like ISO BMFF, then the video data can be stored and transmitted, but efficient access to spatial tiles and regions of interest is not enabled, leading to data duplication and increased decoding requirements
Solution Approach 1:
The patent segments the video bitstream into independent spatial tiles that can be separately accessed and decoded. Each tile is organized as an independent unit within the media segment file, allowing clients to request only the specific tiles corresponding to their region of interest without downloading the entire video stream, thereby eliminating data duplication.
Solution Approach 2:
The patent introduces a new dimensional organization of video data by spatial tiles alongside the traditional temporal dimension. This allows access to video data not only by time (frames) but also by space (tiles), enabling efficient region-of-interest access through a two-dimensional addressing scheme that prevents unnecessary data transmission.
2Productivity
If complete frames are transmitted as contiguous temporal samples, then the video stream maintains temporal coherence, but user-selected regions of interest cannot be efficiently accessed without downloading entire frames
Solution Approach 1:
The patent divides each frame into multiple independent spatial tiles that can be independently decoded and transmitted. This segmentation allows the streaming system to transmit only the specific tiles corresponding to user-selected regions of interest rather than complete frames, improving streaming efficiency by reducing the amount of transmitted data while maintaining temporal coherence through proper timing metadata.
3Ease of operation
If traditional encapsulation methods are used, then file format compatibility is maintained, but parsing efficiency is reduced due to separate handling of geometric and identification information
Solution Approach 1:
The patent merges the geometric information (tile positions, dimensions) and identification information (tile identifiers, layer assignments) into a unified data structure within the media segment file. This consolidation eliminates the need for separate indexation processes and reduces parsing complexity, as the decoder can directly access tile information without cross-referencing multiple separate data structures, thereby improving parsing efficiency while maintaining MPEG standard compatibility.
Data Source
Figure 1a~1b
Figure 2~12
Figure 3a~3c
AI summary
The invention relates to a method for generating a media file based on media data including one or more samples each of which comprises a plurality of tile regions, characterized in that the method comprises generating a plurality of tracks each containing media data corresponding to at least one tile region of the plurality of tile regions corresponding to one sample; generating a media file including all of (i) the plurality of tracks each containing the media data corresponding to the at least one tile region, (ii) a plurality of first boxes for describing metadata of a corresponding track of the plurality of tracks, and (iii) a second box for describing common metadata for the plurality of tracks, wherein each of the first boxes contains 4-letter code which is representing, as a sample entry type for the track, HEVC tile track, and wherein the second box does not contain said 4-letter code, and wherein the metadata in at least one of the plurality of first boxes and the metadata in the second box is referred by a decoder for decoding the media data contained in at least one track.