Reprojection OLED Display for AR Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality systems face challenges in maintaining the stability and correct alignment of virtual objects within the real-world environment due to latency issues, causing virtual objects to drift away from their intended locations as the user moves.
Innovation Solution
The system predicts the future pose of a head-mounted display device and generates pre-rendered images based on this prediction, allowing for updated images to be generated and displayed at a higher frequency than the rendering frame rate through homographic transformations and pixel offset adjustments, ensuring accurate alignment of virtual objects with the real-world environment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the system renders images at the minimum rendering frame rate, then the processing time per frame is reduced, but the displayed frame rate is too low to maintain stable alignment of virtual objects with the real-world environment
Solution Approach 1:
The system pre-renders images at the minimum rendering frame rate in advance before they are needed for display. This allows the rendering process to complete ahead of time, creating a buffer of pre-rendered frames that can be displayed at a higher frame rate without requiring real-time processing for each displayed frame.
Solution Approach 2:
The system dynamically adjusts between rendering frame rate and displayed frame rate. It renders at a lower, fixed frame rate but displays at a higher variable frame rate by combining pre-rendered images with real-time pose adjustments, allowing the display to respond dynamically to user movement while maintaining processing efficiency.
2Productivity
If the system uses pre-rendered images with homographic transformations, then the displayed frame rate increases, but the alignment precision between virtual and real-world objects may deteriorate
Solution Approach 1:
The system continuously receives feedback from pose tracking sensors that monitor the user's head position and orientation. This real-time pose information is used to adjust and update the pre-rendered images through homographic transformations, ensuring that the virtual objects remain accurately aligned with the real-world environment despite the use of pre-rendered content.
Solution Approach 2:
The system performs preliminary rendering at lower resolution or with simplified geometry, then applies high-precision homographic transformations based on real-time pose data to achieve accurate alignment. This two-stage approach allows efficient pre-processing followed by precise real-time adjustment.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
Methods for generating and displaying images associated with one or more virtual objects within an augmented reality environment at a frame rate that is greater than a rendering frame rate are described. The rendering frame rate may correspond with the minimum time to render images associated with a pose of a head-mounted display device (HMD). In some embodiments, the HMD may determine a predicted pose associated with a future position and orientation of the HMD, generate a pre-rendered image based on the predicted pose, determine an updated pose associated with the HMD subsequent to generating the pre-rendered image, generate an updated image based on the updated pose and the pre-rendered image, and display the updated image on the HMD. The updated image may be generated via a homographic transformation and/or a pixel offset adjustment of the pre-rendered image by circuitry within the display.