Multi-directional Video Tile Streaming Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current coding techniques for multi-directional video communications incur latency and inefficiency when handling moving viewports, as they require retransmitting and re-rendering entire frames, leading to suboptimal image quality and increased latency.
Innovation Solution
The system transmits high-quality coding for the current viewport tiles and low-quality coding for other tiles, allowing the receiver to buffer and locally decode non-viewport tiles, reducing latency by prefetching data and enabling seamless viewport changes without round-trip communication.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If current coding techniques transmit and re-render entire frames for moving viewports, then complete image data is available, but latency increases and image quality deteriorates
Solution Approach 1:
The patent divides the multi-directional video frame into multiple tiles, allowing selective transmission and decoding of only the tiles corresponding to the current viewport. This segmentation enables the system to avoid transmitting entire frames when only portions are needed, reducing latency while maintaining image quality for the visible region.
Solution Approach 2:
The patent applies different coding qualities to different tiles based on their relevance to the current viewport. High-quality coding is applied to tiles within the viewport while lower-quality coding is applied to other tiles, optimizing the balance between image quality and bandwidth usage. This local differentiation allows immediate rendering of critical viewport regions without waiting for complete frame retransmission.
2Productivity
If the system retransmits entire frames when viewport changes, then complete data is available, but transmission efficiency decreases
Solution Approach 1:
The patent extracts only the necessary tile data corresponding to the current viewport from the complete frame, rather than retransmitting the entire frame. This extraction approach removes unnecessary data transmission for regions outside the viewport, significantly improving transmission efficiency and reducing the time required to update the display when viewport changes.
Solution Approach 2:
The system performs preliminary actions by buffering and pre-decoding non-viewport tiles in advance, so that when the viewport changes, the data is already ready for immediate rendering. This preliminary processing eliminates the need for round-trip communication and waiting for complete frame retransmission, thereby reducing latency and improving transmission efficiency.
Data Source
AI summary
Aspects of the present disclosure provide techniques for reducing latency and improving image quality of a viewport extracted from multi-directional video communications. According to such techniques, first streams of coded video data are received from a source. The first streams include coded data for each of a plurality of tiles representing a multi-directional video, where each tile corresponding to a predetermined spatial region of the multi-directional video, and at least one tile of the plurality of tiles in the first streams contains a current viewport location at a receiver. The techniques include decoding the first streams and displaying the tile containing the current viewport location. When the viewport location at the receiver changes to include a new tile of the plurality of tiles, retrieving and decoding first streams for the new tile, displaying the decoded content for the changed viewport location, and transmitting the changed viewport location to the source.