Viewport Chunk Indexing for Low-Latency Live Stream Switching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Viewport-switch latency in viewport-dependent immersive video live streaming using CMAF is high, leading to motion sickness and visual discomfort due to long motion-to-photon latency, and existing solutions waste bandwidth and time with inefficient chunk locating processes.
Innovation Solution
Introduce a new 'cloc' box in the ISO base media file format to optimize chunk locating and downloading, reducing viewport-switch latency by enabling fast and efficient chunk indexing and retrieval.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If traditional chunk locating processes are used in CMAF-based viewport-dependent immersive video live streaming, then video streaming can be delivered, but viewport-switch latency becomes high (20 seconds or longer)
Solution Approach 1:
The patent applies preliminary action by pre-generating and storing indexing information (such as segment indexes and chunk location data) during the video encoding and segmentation phase. This indexing information is embedded in the bitstream, allowing the decoding device to quickly locate chunks without performing time-consuming searches during viewport switching. The index structure is prepared in advance to map viewport regions to corresponding chunk locations, enabling fast retrieval when viewport changes occur.
Solution Approach 2:
The patent uses copying by creating and maintaining an index structure that replicates the hierarchical organization of video data (segments, chunks, tiles, and viewport regions). This index acts as a simplified copy or representation of the actual video chunk locations, allowing the system to quickly determine which chunks to download based on the current viewport without having to search through the entire video stream or parse segment structures in real-time.
2Adaptability or versatility
If viewport switching is implemented in real-time, then user interaction and adaptability improve, but motion-to-photon latency increases causing motion sickness and visual discomfort
Solution Approach 1:
The system performs preliminary actions by pre-organizing video content into a hierarchical structure (segments, chunks, tiles) with associated indexing information that maps spatial regions to data locations. When viewport switching occurs, the decoded video data and index structures are already prepared, allowing the system to immediately retrieve and display the new viewport content without delayed processing or searching, thus reducing motion-to-photon latency.
Solution Approach 2:
The patent introduces an intermediary indexing structure that acts as a mediator between the viewport switching request and the actual chunk retrieval process. This index structure translates viewport coordinates into chunk location information, enabling fast and direct access to the required video data without complex real-time searches, thereby reducing the latency between user head movement and visual feedback.
3Measurement precision
If comprehensive chunk information is downloaded and parsed to locate chunks, then accurate chunk retrieval is achieved, but bandwidth is wasted and processing time increases
Solution Approach 1:
The patent extracts only the essential indexing information (segment indexes, chunk location data, and spatial mappings) from the complete video stream and stores it in a compact format within the bitstream. When viewport switching occurs, the system downloads and parses only this extracted index information rather than comprehensive chunk data, enabling accurate chunk location determination with minimal bandwidth consumption and processing overhead.
Solution Approach 2:
The system creates a simplified copy or representation of the video chunk structure through indexing information that contains only the necessary location and identification data. This index copy allows the system to accurately locate chunks without downloading or processing the actual video chunk data, significantly reducing bandwidth usage while maintaining precise chunk location accuracy through the reference information stored in the index.
Data Source
AI summary
This disclosure describes systems, methods, and devices related to enhanced viewport switching. A device may determine a view point change from a first segment to a second segment of a video stream, wherein each segment comprises a plurality of video chunks. The device may download a chunk location box at an end of each video chunk of the plurality of video chunks. The device may parse the chunk location box to update a chunk indexing table. The device may perform a completion check on a video chunk of the plurality of video chunks.