Random-Access Video Stream Prioritization for AR/VR Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding technologies face challenges in providing seamless high-quality augmented reality (AR), virtual reality (VR), and immersive video experiences due to significant time lags when streaming and rendering high-quality image content, especially when the viewer changes positions and orientations, as they require vast amounts of video data and traditional processing orders do not accommodate real-time view direction changes.
Innovation Solution
The implementation of random-access processing orders for video streams that dynamically adjust based on the viewer's real-time view direction, allowing for direct access and prioritization of image blocks corresponding to the viewer's current view direction, incorporating additional metadata like state variables and neighboring block metrics to ensure efficient rendering with minimal latency and artifacts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional video coding standards are used to stream high quality video content, then video quality is maintained, but significant time lags occur when viewer changes positions and orientations
Solution Approach 1:
The video stream is divided into multiple image blocks that can be independently processed and transmitted. This segmentation allows the system to prioritize and transmit only the relevant image blocks corresponding to the viewer's current view direction, rather than transmitting the entire video stream in sequential order, thereby reducing time lag while maintaining video quality.
Solution Approach 2:
The system pre-processes and prepares multiple possible image block sequences in advance, organizing them in a data structure that enables rapid access. When the viewer's view direction changes, the system can immediately retrieve and transmit the pre-prepared relevant image blocks without waiting for sequential processing to complete, thus eliminating perceptible time lags.
2Reliability
If vast amounts of video data are transmitted to support all possible view positions and orientations, then seamless high quality AR/VR experience is achieved, but the data quantity becomes enormous and impractical to support
Solution Approach 1:
Instead of transmitting high-quality video data for all possible view directions simultaneously, the system transmits high-quality data only for the specific image blocks corresponding to the viewer's current view direction. Other regions are either not transmitted or transmitted at lower quality, significantly reducing the total data volume while maintaining seamless experience for the active viewpoint.
Solution Approach 2:
The system transmits slightly more image blocks than strictly necessary (excessive action) by preparing multiple possible view directions in advance, but far less than transmitting all possible views (partial action). This approach ensures seamless transitions when view direction changes while keeping data volume practical, as only a limited set of pre-defined view directions are supported rather than all possible orientations.
3Ease of manufacture
If traditional sequential processing order is used for video streams, then processing simplicity is maintained, but real-time view direction changes cannot be accommodated
Solution Approach 1:
The system uses a dynamic data structure (such as a binary tree or hierarchical structure) that allows the processing order to adapt in real-time based on the viewer's view direction. Image blocks are organized in a way that enables rapid navigation to relevant blocks, and the processing sequence is dynamically adjusted to prioritize blocks corresponding to the current viewpoint, maintaining both adaptability and reasonable processing simplicity.
Data Source
AI summary
A non-random-access video stream is received. A first image block is encoded after second image blocks according to a non-random-access processing order. View direction data is received to indicate a viewer's view direction coinciding with a location covered by the first image block. The first image block is encoded into the random-access video stream before the second image blocks in a random-access processing order. The random-access video stream is delivered to a recipient decoding device operated by the viewer to cause the first image block to be processed and rendered before the second image blocks according to the random-access processing order.