Depth Map Enhancement via Luminance-Guided Upsampling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for generating motion vector fields and depth maps often result in lower resolution outputs, leading to visible artifacts and suboptimal quality, especially when used for high-quality rendering in 3D television and multi-view rendering applications.
Innovation Solution
A method that enhances depth maps by downscaling and upsampling using luminance or color information as a reference, allowing for improved detail reintroduction and noise reduction, while maintaining the same resolution, thereby improving the quality of depth maps and motion vector fields.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If block-based motion estimation is used to generate motion vector fields, then computational complexity is reduced, but the resolution of the output becomes too coarse resulting in visible artifacts
Solution Approach 1:
The image is divided into multiple blocks for motion estimation, where each block generates motion vectors. This segmentation approach allows computational complexity to be managed while the patent subsequently uses these block-based vectors as input for generating higher resolution depth maps through edge detection and refinement processes
Solution Approach 2:
The patent transitions from 2D block-based motion vectors to a multi-dimensional approach by detecting edges in multiple directions (horizontal, vertical, diagonal) and combining them to generate high-resolution depth maps, effectively adding dimensional complexity to overcome the resolution limitation
2Manufacturing precision
If full resolution depth map estimation is generated using edge detection, then depth map quality is improved, but computational cost increases
Solution Approach 1:
Motion vectors are estimated at a coarser block level before generating the final high-resolution depth map. This preliminary action provides a rough depth estimate that guides subsequent edge detection and refinement processes, reducing the overall computational burden compared to direct full-resolution estimation
Solution Approach 2:
The patent applies different processing strategies to different regions of the image: edge detection and refinement are applied selectively at locations where edges are detected, while flat regions use simpler interpolation methods, optimizing computational resources based on local image characteristics
3Speed
If motion vectors are calculated for 8x8 pixel grids, then processing speed is improved, but the resulting depth map has insufficient resolution for high quality rendering
Solution Approach 1:
The image is segmented into 8x8 pixel blocks for rapid motion vector calculation, maintaining processing speed. The patent then uses these block-based vectors as input for a subsequent stage that detects edges and generates high-resolution depth maps, effectively separating the speed-critical and quality-critical processing stages
Solution Approach 2:
The patent compensates for the loss of spatial resolution in block-based vectors by introducing edge direction information (horizontal, vertical, diagonal edges) as an additional dimension of detail, allowing reconstruction of high-resolution depth structures from coarse motion data
Data Source
Figure 1A~1D
Figure 2A
Figure 2B
AI summary
The present invention relates to an image enhancement unit and a method of enhancing a first structure (S1) of samples into a second structure (S2) of samples, the first and the second structure both representing a first property of a scene and having a first resolution, based on a third structure (S3) of samples representing a second property and having the first resolution, the first property and the second property respectively representing different properties of substantially the same scene. The method comprising generating a fourth structure (S4) of samples representing the first property, the fourth structure (S4) of samples having a second resolution lower than the first resolution, by down- scaling first samples of the first structure (S1) of samples to form the samples of the fourth structure (S4) of samples. The method further comprising up-scaling the fourth structure (S4) of samples representing the first property, into the second structure (S2) based on the third structure (S3) of samples, the up-scaling comprising assigning weight factors to respective samples of the fourth structure (S4) of samples based on samples of the third structure (S3) of samples; and computing samples of the second structure (S2) of samples using the samples of the fourth structure (S4) of samples and their respectively assigned weight factors. The invention further relates to an image-processing unit comprising an image enhancement unit according to the invention as well as to a computer program product.