Frame-by-Frame AR/VR Rendering for Low-Latency Head Tracking

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing virtual and augmented reality systems face challenges with high latency and inaccurate head-tracking, leading to inconsistencies between the user's visual and vestibular systems, causing motion sickness and unstable virtual object placement relative to the real world.

Innovation Solution

Implement predictive head tracking and dynamic frame adjustment based on user-specific historical attributes and real-time head movement data to accurately position and render virtual objects, compensating for head movements and reducing latency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If head-tracking is implemented in virtual reality systems, then user immersion is improved, but system latency increases causing motion sickness

Engineering Contradiction:

Improveuser immersionVSAvoidsystem latency

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system performs preliminary head movement prediction using historical head pose data and machine learning models before the actual head movement occurs. This allows the rendering system to pre-calculate and prepare frame data in advance, reducing the effective latency experienced by the user while maintaining accurate head-tracking for immersion.

Inventive Principle:

Principle #10Preliminary action

2Measurement precision

If head movement prediction is implemented, then rendering accuracy is improved, but computational complexity increases

Engineering Contradiction:

Improverendering accuracyVSAvoidcomputational complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The system dynamically adjusts prediction parameters such as the time window for historical data analysis, the complexity of the machine learning model, and the prediction horizon based on current system conditions, user behavior patterns, and performance metrics. This allows the system to optimize the balance between rendering accuracy and computational complexity in real-time.

Inventive Principle:

Principle #35Parameter changes

3Stability of the object's composition

If frame presentation is dynamically adjusted based on head movement, then virtual object stability is improved, but processing time increases

Engineering Contradiction:

Improvevirtual object stabilityVSAvoidprocessing time

Core Design Contradiction:

Stability of the object's compositionVSLoss of time

Solution Approach 1:

The system pre-renders multiple potential frame variations based on predicted head movement trajectories and stores them in buffers. When head movement is detected, the system selectively presents the pre-prepared appropriate frame rather than rendering in real-time, thereby maintaining virtual object stability while minimizing processing time during actual interaction.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS12380662B2Frame-by-frame rendering for augmented or virtual reality systems

Publication Date: 2025.08.05 MAGIC LEAP INC

AI summary

One embodiment is directed to a user display device comprising a housing frame mountable on the head of the user, a lens mountable on the housing frame and a projection sub system coupled to the housing frame to determine a location of appearance of a display object in a field of view of the user based at least in part on at least one of a detection of a head movement of the user and a prediction of a head movement of the user, and to project the display object to the user based on the determined location of appearance of the display object.