AR Display Latency Compensation via Predictive Warp Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Head-up displays using augmented reality technology experience latency issues, leading to position shifts between virtual and real objects due to delays in sensor data processing and rendering, which are not accurately reflected in the display position, causing misalignment and distortion.
Innovation Solution
A display system that incorporates a tracking processing unit, rendering image generation unit, and warp processing unit, which uses latency compensation parameters to correct warp parameters in real-time, minimizing position shifts by performing latency compensation during the warp processing stage closest to the display timing, thus ensuring accurate alignment of virtual objects with real objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If sensor output is acquired and rendering is performed using the sensor output, then virtual objects can be displayed, but a time lag occurs between sensor sampling and display, causing position shifts between virtual and real objects
Solution Approach 1:
The patent applies preliminary action by predicting the position and orientation of the mobile body at the display timing based on sensor outputs acquired at earlier timings. The prediction unit calculates future states before display occurs, allowing the system to pre-compensate for latency. This enables the virtual object to be rendered at the predicted future position rather than the past sampled position, resolving the alignment accuracy issue caused by time lag.
Solution Approach 2:
The patent implements feedback by continuously acquiring sensor outputs and using them to update predictions. The system creates a closed-loop control where sensor data feeds into the prediction mechanism, which then adjusts the virtual object positioning. This continuous feedback loop ensures that predictions are constantly refined based on actual sensor measurements, maintaining alignment accuracy despite ongoing latency.
2Manufacturing precision
If warp processing is performed on rendering images, then distortion can be corrected, but position shifts due to latency are not reflected in the display position
Solution Approach 1:
The patent applies preliminary action by performing prediction and latency compensation before the warp processing stage. The prediction unit calculates future position and orientation, and this predicted information is then used as input for warp processing. By preparing the corrected position data in advance, the system ensures that when warp processing occurs, it operates on already-compensated data, eliminating the need for additional time-consuming correction steps during display.
Solution Approach 2:
The patent introduces an intermediary prediction mechanism between sensor acquisition and warp processing. This prediction unit acts as a mediator that translates past sensor data into future state predictions, which then feed into the warp processing stage. This intermediary layer allows the system to decouple the timing of sensor sampling from display timing, enabling accurate distortion correction that accounts for latency without extending processing time.
3Measurement precision
If multiple correction stages are implemented, then position accuracy can be improved, but system complexity increases
Solution Approach 1:
The patent merges the prediction function with the existing warp processing unit. Rather than adding a separate prediction correction unit that would increase system complexity, the prediction functionality is integrated into the warping process. The prediction unit works closely with the warp processing unit, combining their functions to achieve both position prediction and distortion correction in a unified workflow, thereby improving accuracy without proportionally increasing complexity.
Solution Approach 2:
The patent makes the warp processing unit multi-functional by having it perform both traditional distortion correction and latency compensation based on predicted positions. Instead of creating dedicated units for each correction type, the warp processing unit is enhanced to handle multiple correction objectives simultaneously. This universal approach allows a single unit to achieve multiple correction goals, reducing overall system complexity while maintaining high measurement precision.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A display system (100) includes a rendering image generation unit (120), a parameter computation unit (140), and a warp processing unit (130). The parameter computation unit (140) computes a latency compensation parameter for compensating a latency including a rendering processing latency of a rendering image based on tracking information. The warp processing unit (130) performs latency compensation processing for compensating the position of a virtual object in a display region based on the latency compensation parameter.