Iterative Image Reprojection for Real-Time Stereoscopic Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current image space reprojection techniques, such as depth-image-based rendering, are complex and time-consuming, and fail to handle disocclusion regions effectively, making real-time rendering of three-dimensional content challenging, especially for high-performance graphics applications.
Innovation Solution
An iterative fixed-point reprojection algorithm is employed, using an insignificant parallax threshold depth to uniformly shift pixels with high depth values and iteratively converge on corresponding pixels in the source image for those with lower depth values, with an additional rendering pass to fill disoccluded regions, enabling real-time generation of stereoscopic pairs and other spatio-temporal rendering applications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If depth-image-based rendering (DIBR) is used for image space reprojection, then stereoscopic image pairs can be generated, but the computation is complex and time-consuming, preventing real-time rendering
Solution Approach 1:
The patent segments the image processing into two distinct parts: (1) a complex DIBR computation performed offline to generate depth maps and reference images, and (2) a simplified real-time reprojection that uses pre-computed data structures (depth buffer, disparity buffer, occlusion buffer) to rapidly generate stereoscopic pairs. This segmentation allows high-quality rendering without real-time computational burden.
Solution Approach 2:
The patent performs preliminary actions by pre-computing depth maps, disparity buffers, and occlusion buffers during an offline DIBR phase. These pre-computed data structures are stored and reused during real-time rendering, eliminating the need for complex computations during the actual rendering process and enabling real-time performance.
2Reliability
If traditional DIBR techniques are used, then image reprojection can be achieved, but disocclusion regions cannot be properly handled
Solution Approach 1:
The patent applies local quality by treating different regions of the image differently: (1) visible regions use standard reprojection from the depth buffer, (2) disoccluded regions are detected using the occlusion buffer and handled separately by copying from the other eye's image or using background modeling. This localized approach ensures accurate handling of disocclusion regions while maintaining overall reprojection quality.
Solution Approach 2:
The patent introduces an occlusion buffer as an intermediary data structure that tracks which pixels are disoccluded during reprojection. This buffer acts as a mediator between the reprojection process and the final image composition, allowing the system to identify and specially process disoccluded regions without affecting the overall reprojection workflow.
3Reliability
If rendering both images in the stereographic pair is performed, then complete three-dimensional effect can be achieved, but real-time processing is not possible for high-performance graphics applications
Solution Approach 1:
The patent uses copying by reusing the rendered source image and pre-computed depth/disparity buffers to generate the target stereoscopic image through pixel sampling and warping operations. Instead of rendering both images from scratch, the system copies and transforms pixel data from the source view to the target view using the pre-computed depth information, achieving real-time performance while maintaining quality.
Data Source
AI summary
Techniques are disclosed for performing image space reprojection iteratively. An insignificant parallax threshold depth is computed for a source image. Portions of the image having depth values greater than the insignificant parallax threshold depth may be shifted uniformly to produce corresponding portions of the reprojection (target) image. An iterative fixed-point reprojection algorithm is used to reproject the portions of the source image having depth values less than or equal to the insignificant parallax threshold depth. The fixed point reprojection algorithm quickly converges on the best pixel in the source image for each pixel in a target image representing an offset view of the source image. An additional rendering pass is employed to fill disoccluded regions of the target image, where the reprojection algorithm fails to converge.