Foveated Simulation for VR Gaze Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current virtual reality and augmented reality applications face challenges in running complex applications at high fidelity due to high computational demands, even with foveated rendering, which cannot efficiently utilize processing resources to enhance visual quality and reduce motion sickness.
Innovation Solution
The implementation of foveated simulation, which reduces computational resources outside the user's gaze fixation point by determining the gaze fixation point using a gaze tracking system, allowing for selective adjustment of simulation fidelity and resource allocation to increase visual frame rate and immersion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If foveated rendering is used to concentrate detail near the center of vision, then processing resources are reduced, but complex virtual reality applications cannot run at high fidelity
Solution Approach 1:
The patent applies local quality by implementing different simulation fidelity levels in different spatial regions relative to the user's gaze point. High-fidelity simulation is concentrated in the foveal region (center of gaze) where visual acuity is highest, while peripheral regions use lower-fidelity simulation. This resolves the contradiction by allocating processing resources locally according to visual importance, maintaining high fidelity where needed while reducing overall computational load.
Solution Approach 2:
The patent segments the visual field into multiple regions (foveal, intermediate, and peripheral zones) based on distance from the gaze point. Each region is assigned a different simulation fidelity level, allowing the system to process complex simulations only in the small foveal region while using simplified simulations in larger peripheral regions. This segmentation enables high fidelity in critical areas without requiring full high-fidelity processing across the entire field of view.
2Manufacturing precision
If high fidelity simulation is maintained across the entire field of view, then simulation accuracy is improved, but computational load increases and frame rate decreases
Solution Approach 1:
The patent implements local quality by varying simulation accuracy locally across the visual field. The foveal region maintains high simulation accuracy to preserve visual quality where the user is looking, while peripheral regions use reduced accuracy simulation. This local differentiation maintains overall visual fidelity and user immersion while reducing the total computational load enough to achieve higher frame rates.
Solution Approach 2:
The patent applies dynamics by making simulation fidelity adaptive and dynamic rather than static. The system continuously tracks the user's gaze and dynamically adjusts which regions receive high-fidelity simulation processing. As the user moves their eyes, the high-fidelity simulation region moves with it, ensuring that computationally expensive accurate simulation is always applied to the current foveal region while maintaining high frame rates through selective processing.
3Productivity
If computational resources are reduced outside the gaze fixation point, then frame rate increases, but visual quality outside fixation deteriorates
Solution Approach 1:
The patent resolves this contradiction by applying local quality principles where visual quality is maintained at high levels in the foveal region (center of gaze) while accepting reduced quality in peripheral regions. This is appropriate because human vision naturally prioritizes the center of gaze, so the differential quality matches human perceptual characteristics. The frame rate improvement comes from reducing processing in peripheral regions where high quality is less critical.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A system includes a head mounted display (HMD), a processor, and a memory. The HMD is configured to generate sensor data indicative of a gaze vector of a user and a position of the user. The processor is configured to communicate with the HMD. The memory is coupled to the processor and stores instructions that, when executed by the processor, cause the processor to obtain data indicating the gaze vector of the user and the position of the user. The instructions also cause the processor to determine a component vector for a virtual component of a virtual reality simulator based on the position of the user and a position of the virtual component. The instructions further cause the processor to calculate an alignment value based on the gaze vector and the component vector, and adjust a simulation setting of the virtual component based on the alignment value.