Foveated Rendering Streams for Reliable Low-Latency AR/VR Delivery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AR/VR headset systems face challenges in providing immersive experiences with efficient image and video rendering, particularly in reducing latency, jitter, and improving bandwidth consumption while maintaining high-quality display content.
Innovation Solution
Implement foveated rendering techniques that separate video traffic into high acuity (HA) and low acuity (LA) streams, prioritize HA traffic with differentiated services, and use modulation coding schemes and Wi-Fi Multimedia protocols to enhance reliability and reduce latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If foveated rendering is implemented to reduce rendering workload, then bandwidth consumption is reduced, but reliability of high acuity traffic may deteriorate
Solution Approach 1:
The patent segments video traffic into high acuity (HA) and low acuity (LA) streams based on foveated rendering principles. HA traffic contains critical visual information requiring high reliability transmission, while LA traffic contains peripheral information that can tolerate lower reliability. This segmentation allows differential treatment of traffic streams to optimize both bandwidth efficiency and reliability.
Solution Approach 2:
The patent applies local quality by transmitting HA traffic with higher reliability parameters (lower modulation coding schemes, higher redundancy) compared to LA traffic. This ensures that critical foveal region content maintains high quality and reliability while peripheral content consumes less bandwidth, resolving the contradiction between overall bandwidth efficiency and HA traffic reliability.
2Manufacturing precision
If HA traffic is prioritized with higher reliability parameters, then displayed content quality is improved, but bandwidth consumption increases
Solution Approach 1:
The patent applies partial action by providing high reliability transmission only for the necessary HA traffic portion, rather than treating all traffic equally. By applying enhanced reliability parameters selectively to HA streams based on foveated rendering requirements, the system achieves high displayed content quality for critical regions without the excessive bandwidth consumption that would result from uniformly high reliability transmission across all content.
3Ease of operation
If foveated rendering streams are transmitted with differential reliability, then user experience is improved, but device complexity increases
Solution Approach 1:
The patent introduces an intermediary classification mechanism that automatically separates traffic into HA and LA streams based on foveated rendering analysis. This intermediary process handles the complexity of differential reliability management, allowing the system to improve user experience through optimized content delivery without requiring complex manual configuration or processing at endpoint devices.
Data Source
AI summary
Methods and systems are described for improved delivery of foveated rendering streams for media content delivery. Received video data is separated into a high acuity (HA) stream and a low acuity (LA) stream. The HA stream is marked to indicate HA transmission. A communication device that retransmits the HA stream based on the marking transmits the HA stream at a lower latency and a higher reliability than the LA stream. A receiving device receives the HA stream and the LA stream. A display device generates for display a combined stream including the HA stream and the LA stream. Artificial intelligence systems, including neural networks, are trained for improving the foveated rendering. Models are developed for the foveated rendering. Related apparatuses, devices, techniques, and articles are also described.