Predictive Foveated VR System Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional virtual reality and augmented reality systems suffer from latency issues, leading to eyestrain, headaches, and nausea due to significant lag times and high image data requirements, which increase system cost and size.
Innovation Solution
A predictive, foveated virtual reality system that captures image data using multiple resolutions, employing lower resolution cameras for a wide field of view and higher resolution cameras for a narrow field of view, and anticipates user movements to prepare and display image data ahead of time, utilizing sensors and gaze tracking modules to determine the user's line of sight vector.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional virtual reality systems capture and display high resolution image data for the entire field of view, then the user experiences a complete view of the virtual environment, but the system suffers from increased latency and higher computational overhead
Solution Approach 1:
The patent applies local quality by capturing high resolution image data only in the foveal region (central field of view where user attention is focused) while using lower resolution for peripheral regions. This selective resolution approach reduces overall data processing requirements and latency while maintaining perceived visual quality where it matters most to the user experience
Solution Approach 2:
The patent segments the field of view into multiple regions including the foveal region and peripheral regions, processing each region at appropriate resolution levels. This segmentation allows the system to prioritize computational resources for the central view while reducing overhead for peripheral areas, thereby decreasing overall latency
2Measurement precision
If conventional virtual reality systems capture high resolution image data for the entire field of view, then the user experiences a complete view of the virtual environment, but the system cost and size increase
Solution Approach 1:
The patent implements local quality by deploying high resolution cameras only for the foveal region rather than uniformly across the entire field of view. This reduces the total number of high resolution sensors required, thereby decreasing system cost and physical size while maintaining adequate resolution where users actually look
Solution Approach 2:
The patent segments the imaging system into multiple camera modules positioned to cover different regions of the field of view, with high resolution cameras targeted at the foveal region and lower resolution coverage for peripheral areas. This segmented approach reduces overall system complexity and cost
3Ease of operation
If conventional virtual reality systems process image data in real-time without prediction, then the system responds to user movements, but significant lag time occurs between user action and system response
Solution Approach 1:
The patent applies preliminary action by using gaze tracking and motion prediction algorithms to anticipate where the user will look next, pre-processing and preparing image data for the predicted foveal region before the user actually looks there. This predictive preprocessing reduces the effective latency between user movement and system response
4Productivity
If the system uses multiple resolution cameras for foveated rendering, then computational overhead is reduced, but the system must accurately determine the user's line of sight
Solution Approach 1:
The patent introduces intermediary components including gaze tracking cameras and motion sensors that mediate between the user's physical movements and the virtual reality rendering system. These intermediaries provide data for predicting the user's line of sight, enabling accurate foveated rendering without requiring direct measurement of eye position
Data Source
AI summary
A Predictive, Foveated Virtual Reality System may capture views of the world around a user using multiple resolutions. The Predictive, Foveated Virtual Reality System may include one or more cameras configured to capture lower resolution image data for a peripheral field of view while capturing higher resolution image data for a narrow field of view corresponding to a user's line of sight. Additionally, the Predictive, Foveated Virtual Reality System may also include one or more sensors or other mechanisms, such as gaze tracking modules or accelerometers, to detect or track motion. A Predictive, Foveated Virtual Reality System may also predict, based on a user's head and eye motion, the user's future line of sight and may capture image data corresponding to a predicted line of sight. When the user subsequently looks in that direction the system may display the previously captured (and augmented) view.