Selective XR Object Warping for Latency-Induced Display Distortion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Movement of physical objects in extended reality environments can introduce display distortion due to latency and large correction magnitudes in the reprojection of computer-generated content, particularly in rotational movements.
Innovation Solution
Implement late stage selective warping of computer-generated objects based on predicted poses of physical objects, considering translational and rotational movements independently, to prevent display distortion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If standard reprojection is used to update computer-generated content position, then the content follows physical object movement, but display distortion occurs due to latency and large correction magnitudes
Solution Approach 1:
The system performs preliminary warping of the computer-generated object based on a predicted pose of the physical object at a future time point. By anticipating the movement and pre-distorting the content in the opposite direction, the system compensates for latency-induced positioning errors and prevents display distortion when the object is actually displayed at the predicted time
Solution Approach 2:
The warping operation is applied selectively to specific portions of the computer-generated object based on their distance from the predicted pose. Portions closer to the predicted position undergo less warping, while distant portions undergo more warping, creating a gradient that maintains visual coherence and reduces overall distortion
2Speed
If reprojection with large correction magnitude is performed, then content can catch up with physical object movement, but display distortion is introduced
Solution Approach 1:
Instead of performing large corrective reprojection operations after latency occurs, the system preliminarily warps the content based on predicted future pose. This spreads the correction over time and reduces the magnitude of any single transformation, preventing the display distortion that results from abrupt large-magnitude corrections
3Object-affected harmful factors
If warping is applied to compensate for latency, then display distortion is reduced, but computational complexity increases
Solution Approach 1:
The system applies warping selectively to different portions of the computer-generated object based on their distance from the predicted pose. This local differentiation allows the system to focus computational resources on the most critical portions of the object while reducing unnecessary computations for portions that require minimal or no warping
Solution Approach 2:
The system applies warping only to the extent necessary to compensate for predicted latency, rather than uniformly warping the entire object. By calculating warping magnitude as a function of distance from the predicted pose, the system performs partial action that is sufficient to prevent distortion while minimizing unnecessary computational overhead
Data Source
AI summary
A method is performed at a first electronic device with one or more processors, a non-transitory memory, and a display. The method includes rendering a computer-generated object at a first time. The computer-generated object is positionally dependent on a first physical object. The method includes obtaining a predicted pose of the first physical object at a second time. The second time is after the first time. The method includes warping a first portion of the computer-generated object based at least in part on a first distance between the predicted pose and the first portion. The method includes displaying, on the display at the second time, the warped first portion of the computer-generated object.