Saccade-Aware AR/VR Rendering for Power and Compute Savings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
High computational and power resource demands in head-mounted AR/VR devices due to high-resolution and high-frame-rate rendering requirements, which are typically limited in these devices.
Innovation Solution
Adjust rendering quality by reducing resolution and/or frame rate during a post-saccade window following a user's gaze change, with adjustable duration and potentially dynamic ramp-up, leveraging saccade detection and eye tracking.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If high-resolution and high-frame-rate rendering is used, then rendering quality is improved, but power consumption and computational resource demands increase
Solution Approach 1:
The system dynamically adjusts rendering resolution and frame rate based on real-time saccade detection. During saccadic eye movements, rendering quality is temporarily reduced, and between saccades, full quality rendering is restored. This dynamic adaptation resolves the contradiction by making rendering quality variable rather than static, matching the visual system's temporal sensitivity.
Solution Approach 2:
The system implements periodic rendering quality adjustment synchronized with the natural periodicity of saccadic eye movements. Rendering is alternated between high quality (during fixation periods) and reduced quality (during saccade periods), creating a rhythmic pattern that exploits the periodic nature of human visual sampling to reduce overall computational load while maintaining perceived quality.
2Manufacturing precision
If high-resolution rendering is used, then image quality is improved, but computational resource demands increase
Solution Approach 1:
The system dynamically adjusts rendering resolution based on real-time saccade detection. During saccadic eye movements, rendering resolution is temporarily reduced, and between saccades, full quality rendering is restored. This dynamic adaptation resolves the contradiction by making rendering quality variable rather than static, matching the visual system's temporal sensitivity.
Solution Approach 2:
The system changes the rendering parameter (resolution) based on the detected eye movement state. When a saccade is detected, the rendering resolution parameter is temporarily reduced; when fixation is detected, the parameter is restored to full quality. This parameter modulation resolves the contradiction by adapting the computational demand to the actual visual processing requirements.
3Reliability
If rendering quality is continuously maintained at high levels, then visual experience is improved, but power and computational resources are depleted faster
Solution Approach 1:
The system dynamically adjusts rendering quality based on real-time saccade detection. During saccadic eye movements, rendering quality is temporarily reduced, and between saccades, full quality rendering is restored. This dynamic adaptation resolves the contradiction by making rendering quality variable rather than static, matching the visual system's temporal sensitivity.
Solution Approach 2:
The system implements periodic rendering quality adjustment synchronized with the natural periodicity of saccadic eye movements. Rendering is alternated between high quality (during fixation periods) and reduced quality (during saccade periods), creating a rhythmic pattern that exploits the periodic nature of human visual sampling to reduce overall computational load while maintaining perceived quality.
Data Source
AI summary
Augmented and/or virtual reality (AR/VR) near-eye display devices implementing adjustment of rendering quality based on saccade detection to preserve computational and power resources are disclosed. In examples, an eye tracking system for an augmented reality/virtual reality (AR/VR) display device comprises a display to render content, an eye tracking system to capture a user's gaze, and a controller to manage the display and the eye tracking system. The controller may detect a saccade based on capturing a shift in the user's gaze, reduce a rendering quality of the rendered content for a duration of a post-saccade window; and increase the rendering quality upon completion of the post-saccade window.