Stereo Rendering System for Real-Time 3D HMD Output
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current stereo rendering techniques face challenges in generating 3D images and videos in real-time due to the processor-intensive nature of optical flow interpolation, especially with semi-transparent objects and the limitations of existing optical flow algorithms, making real-time 3D rendering difficult.
Innovation Solution
A system and method that captures images using a camera rig with multiple cameras positioned to capture 2D images from different viewpoints, which are then processed to generate left and right eye images without interpolating 3D images, using an image processing system to combine and shift images based on depth of view for sharp rendering on a head-mounted display.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If optical flow interpolation is used to generate 3D images from 2D captures, then 3D image quality is improved, but processing time and computational complexity increase significantly
Solution Approach 1:
The patent segments the 3D image generation process into two distinct phases: (1) capturing multiple 2D images from different viewpoints using a camera rig, and (2) rendering the 3D image by selecting and combining pre-captured images based on depth information. This segmentation eliminates the need for real-time optical flow interpolation while maintaining 3D quality, as the computational burden is shifted to the offline image capture phase.
Solution Approach 2:
The patent performs preliminary action by capturing a complete set of 2D images from multiple viewpoints before the actual 3D rendering is needed. This pre-capture of reference images from various angles allows the rendering system to quickly assemble 3D images without performing intensive interpolation calculations in real-time, thus reducing processing time while maintaining image quality.
2Productivity
If optical flow algorithms are used for real-time 3D rendering, then 3D video generation speed is improved, but computational resources and processing power are excessively consumed
Solution Approach 1:
The patent divides the system into a capture subsystem (camera rig) and a rendering subsystem. The capture subsystem pre-acquires images from multiple viewpoints, while the rendering subsystem simply selects and combines these pre-captured images based on depth maps. This segmentation transfers the computational workload from the rendering phase to the capture phase, enabling real-time rendering with minimal computational resources.
Solution Approach 2:
The patent uses multiple cameras to capture copies of the scene from different viewpoints simultaneously. These copied images serve as reference material for rendering 3D images without requiring any interpolation or heavy computation. The system essentially copies the visual information from multiple angles and recombines them, avoiding the need for computationally intensive optical flow algorithms.
3Manufacturing precision
If multiple cameras are used to capture images from different viewpoints, then 3D rendering accuracy is improved, but device complexity increases
Solution Approach 1:
The patent merges multiple camera systems into a single integrated camera rig that captures images from multiple viewpoints simultaneously. The cameras are positioned at known spatial relationships and synchronized to capture the same scene from different angles. This merging approach achieves accurate 3D rendering through geometric diversity rather than through complex individual camera mechanisms, simplifying the overall system while maintaining precision.
Data Source
Figure 1
Figure 2
Figure 3A~3B
AI summary
A method includes receiving an indication of a field of view associated with a three- dimensional (3D) image being displayed on a head mount display (HMD), receiving an indication of a depth of view associated with the 3D image being displayed on the HMD, selecting a first right eye image and a second right eye image based on the field of view, combining the first right eye image and the second right eye image based on the depth of view, selecting a first left eye image and a second left eye image based on the field of view, and combining the first left eye image and the second left eye image based on the depth of view.