Wide Baseline Stereo Rendering for Low-Latency VR
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current virtual and augmented reality systems face challenges in accurately tracking head movements and reducing latency, leading to motion sickness and unstable virtual object placement due to high latency and parallax warping issues, which result in holes in rendered images.
Innovation Solution
The system renders and warps left and right synthetic images from wider focal centers spaced greater than the inter-ocular distance, then displays them at narrower focal centers aligned with the inter-ocular distance, using depth data and predictive head position estimation to minimize latency and hole frequency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional stereo rendering with inter-ocular distance focal centers is used, then image stability is maintained, but latency increases and motion sickness occurs
Solution Approach 1:
The system pre-renders images at wider focal centers before head movement occurs, allowing the rendering pipeline to complete faster and reduce latency. By preparing images at exaggerated focal points in advance, the system can quickly warp them to match actual head position without waiting for slow conventional rendering
Solution Approach 2:
The system changes the focal center parameter from conventional inter-ocular distance to wider spaced focal centers during rendering. This parameter modification allows for faster image generation that can be subsequently warped to correct focal points, effectively trading rendering speed for image quality
2Measurement precision
If head tracking is updated at high rate, then motion accuracy improves, but computational complexity and processing load increase
Solution Approach 1:
The system pre-calculates warp parameters and depth data structures in advance, so that when head movement data arrives, the complex computational work has already been prepared. This preliminary preparation reduces the real-time processing burden while maintaining high tracking accuracy
Solution Approach 2:
The system extracts and pre-processes the computationally intensive depth calculation and warp parameter generation into separate preprocessing steps, removing the heavy computational load from the real-time rendering pipeline and allowing high-rate head tracking without overwhelming the processor
3Manufacturing precision
If parallax warping is applied to correct focal centers, then image quality improves, but holes and artifacts appear in rendered images
Solution Approach 1:
The system pre-renders images at wider focal centers that capture more of the scene geometry and depth information before warping. By having excess visual information available in the pre-rendered images, the subsequent warping operation can correct focal centers without creating holes, as the additional peripheral information fills in gaps that would otherwise appear
4Loss of time
If wider focal centers are used for rendering, then latency and hole frequency reduce, but initial image distortion increases
Solution Approach 1:
The system accepts and utilizes the distortion inherent in pre-rendered wide focal center images as a necessary intermediate state. The distortion is corrected in the next step through warping to the proper focal centers, so the initial distortion is temporary and purposeful rather than a final defect
Solution Approach 2:
The system dynamically changes the focal center parameter from wide (for fast rendering) to correct inter-ocular distance (for proper geometry). This parameter transformation converts the distorted but fast-to-render images into correctly proportioned final images, effectively using parameter adjustment to resolve the distortion issue
Data Source
AI summary
A virtual image generation system and method of operating same are provided. A left synthetic image and a right synthetic image of a three-dimensional scene are rendered respectively from a first left focal center and a first right focal center relative to a first viewpoint. The first left and first right focal centers are spaced from each other a distance greater than the inter-ocular distance of an end user. The synthetic image and the right synthetic image are warped respectively to a second left focal center and a second right focal center relative to a second viewpoint different from the first viewpoint. The second left and right focal centers are spaced from each other a distance equal to the inter-ocular distance of the end user. A frame is constructed from the left and right warped synthetic images, and displayed to the end user.