Virtual-Viewpoint Image Generation Using Sub-Pixel Disparity Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for generating virtual-viewpoint images in 3D displays face challenges due to the low accuracy of depth images, leading to noise and deteriorated image quality, especially when using contour information for correction and feature point extraction.
Innovation Solution
The proposed solution involves generating a disparity image using a depth image and a virtual camera parameter, correcting disparity values based on contour lines from both the two-dimensional and disparity images, and interpolating unfixed pixels to reduce noise and enhance image quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a depth image is captured by using an actual stereo camera or TOF camera, then depth information can be obtained, but the resolution of the depth image becomes lower than that of the two-dimensional image
Solution Approach 1:
The patent applies parameter changes by using sub-pixel precision disparity values (with fractional parts) instead of integer pixel values to represent depth information. This allows the depth image to maintain the same resolution as the two-dimensional image while providing more precise depth measurements through the fractional disparity values.
2Speed
If 3D-CG tools sacrifice the resolution of a depth image, then rendering speed increases, but the accuracy of depth information decreases
Solution Approach 1:
The patent changes the parameter representation from integer pixel values to sub-pixel precision values (including fractional parts). This allows maintaining high resolution depth information without sacrificing rendering speed, as the fractional disparity values can be efficiently calculated and processed.
3Measurement precision
If a depth image is estimated from signal of plural two-dimensional images, then depth information can be obtained, but the accuracy becomes less accurate because of misdetecting corresponding pixels
Solution Approach 1:
The patent uses feedback by iteratively refining disparity values through multiple passes. The initial disparity estimation is improved by using the fractional parts to correct mismatches, and this corrected information feeds back into the matching process to achieve more accurate corresponding pixel detection.
Solution Approach 2:
The patent performs preliminary action by first calculating integer pixel disparities and then refining them with sub-pixel precision. This preliminary integer estimation provides a foundation that is subsequently improved through fractional disparity calculation, reducing the impact of initial misdetections.
4Productivity
If conventional methods are used to generate virtual-viewpoint images from low-resolution depth images, then image generation can be performed, but noise becomes conspicuous and image quality deteriorates
Solution Approach 1:
The patent changes the disparity value parameter from integer to sub-pixel precision (including fractional parts). This parameter change enables accurate warping and interpolation operations that reduce noise and improve image quality while maintaining efficient image generation capability.
Data Source
AI summary
Provided are an image generation apparatus, three-dimensional image display apparatus, image generation method and non-transitory computer readable medium. An image generation apparatus for generating a virtual-viewpoint image, includes a disparity-image-generating section which generates a disparity image corresponding to a two-dimensional image, by using a depth image and a parameter of a virtual camera; a disparity-image-correcting section which corrects the disparity image based on contour lines extracted from the two-dimensional image and the disparity image, or corrects the depth image to be used for generating the disparity image based on contour lines extracted from the two-dimensional image and the depth image; an pixel-shift-processing section which shifts each pixel of the two-dimensional image according to the corrected disparity image, to generate an intermediate image; and a unfixed-pixel-area-interpolating section which interpolates unfixed pixels to which the shifted pixels have been assigned in the intermediate image, to generate the virtual-viewpoint image.