Depth Map View Synthesis for Real-Time Multiview Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies struggle to efficiently convert single-view images into multiview images in real-time for multiview displays, often resulting in artifacts, holes, and computational inefficiencies.
Innovation Solution
A method using raycasting with a shader program to synthesize multiview images from a color image and depth map, optimizing ray traversal with predetermined intervals and threshold comparisons to determine pixel values, and employing fragment shaders for interlacing and hole filling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional multiview image generation methods are used, then multiple views can be rendered, but computational complexity increases and real-time performance is难以 to achieve
Solution Approach 1:
The patent pre-calculates and stores depth information in a depth map before view synthesis. By having depth data readily available beforehand, the system avoids performing complex depth calculations during real-time rendering, thus achieving both real-time performance and reduced computational complexity during the actual view generation process
Solution Approach 2:
The patent replaces traditional geometric rendering methods with a shader-based raycasting approach. Instead of using complex geometric intersection calculations, the system uses fragment shaders to simulate ray traversal through the depth map, substituting mechanical geometric computations with more efficient pixel-parallel processing that achieves real-time performance
2Manufacturing precision
If high-quality multiview images are generated, then artifacts and holes are reduced, but computational load increases
Solution Approach 1:
The patent applies hole filling algorithms selectively only to regions where holes are detected in the synthesized view, rather than processing the entire image. This partial action approach maintains image quality by filling only the necessary areas while significantly reducing the overall computational load compared to processing the complete image at full resolution
Solution Approach 2:
The patent introduces a depth map as an intermediary data structure that facilitates accurate view synthesis. By using the depth map to guide ray traversal and pixel selection, the system achieves high image quality with fewer artifacts and holes, while the intermediary depth information enables efficient computation by avoiding the need for complex iterative optimization algorithms
3Measurement precision
If raycasting is used for view synthesis, then accurate pixel values can be determined, but processing time increases
Solution Approach 1:
The patent segments the view synthesis process into distinct shader stages: one fragment shader handles the raycasting and pixel value determination, while another handles interlacing and hole filling. This segmentation allows each stage to be optimized independently and executed in parallel on GPU hardware, maintaining pixel value accuracy while reducing overall processing time through parallelization
Solution Approach 2:
The patent changes the computational parameters by working directly in screen space with fragment coordinates rather than performing raycasting in 3D space. By transforming the problem into 2D fragment processing with depth map sampling, the system maintains accurate pixel values through depth-based sampling while dramatically reducing processing time through efficient GPU fragment shader execution
Data Source
AI summary
In multiview image generation and display, a computing device can synthesize view images of a multiview image of a scene from a color image and a depth map. Each view image can include color values at respective pixel locations. The computing device can render the view images of the multiview image on a multiview display. Synthesizing a view image can include, for a pixel location in the view image, the following operations. The computing device can cast a ray from the pixel location toward the scene in a direction corresponding to a view direction of the view image. The computing device can determine a ray intersection location at which the ray intersects a virtual surface specified by the depth map. The computing device can set a color value of the view image at the pixel location to correspond to a color of the color image at the ray intersection location.