Joint Bilateral Up-Sampling for 3D Depth Data Resolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional three-dimensional image sensors have a lower resolution for depth data compared to two-dimensional color data, limiting their ability to provide high-quality depth information.
Innovation Solution
An image processing system that includes a calculation unit, reconstruction unit, confidence map estimation unit, and up-sampling unit to calculate phase differences, reconstruct light, estimate confidence maps, and perform joint bilateral up-sampling to increase the resolution of three-dimensional depth data to match or exceed the resolution of two-dimensional color data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional three-dimensional image sensors are used to obtain depth information, then depth data can be acquired, but the resolution of depth data is lower than that of two-dimensional color data
Solution Approach 1:
The patent introduces a confidence map as an intermediary element that mediates between the low-resolution depth data and the up-sampling process. The confidence map guides the joint bilateral up-sampling by indicating reliable and unreliable regions, allowing the system to selectively enhance depth information quality without uniformly processing all pixels, thus improving depth data resolution while preserving information integrity.
Solution Approach 2:
The patent changes the parameter of resolution by applying joint bilateral up-sampling with adaptive kernel sizes. The up-sampling process dynamically adjusts the spatial and range kernel parameters based on the confidence map, enabling local adaptation where high-confidence regions receive more aggressive up-sampling while low-confidence regions are preserved, thereby improving overall depth data resolution.
2Measurement precision
If joint bilateral up-sampling is performed without confidence map guidance, then computational complexity increases, but accuracy improvement is limited due to noise propagation
Solution Approach 1:
The patent segments the up-sampling process into two distinct stages: first generating a confidence map that categorizes pixels by reliability, then performing joint bilateral up-sampling guided by this segmentation. This segmentation allows the system to apply different processing strategies to different regions, reducing unnecessary computational complexity in low-confidence areas while focusing computational resources on high-confidence regions for maximum accuracy improvement.
Solution Approach 2:
The patent applies local quality by making the up-sampling process adaptive to local confidence characteristics. The joint bilateral filter uses the confidence map to locally adjust filtering strength and kernel parameters, applying aggressive filtering only where confidence is high and minimal filtering where confidence is low. This local adaptation optimizes the balance between accuracy improvement and computational complexity.
3Productivity
If simple up-sampling methods are used to increase depth data resolution, then processing speed is maintained, but noise is amplified and boundary details are lost
Solution Approach 1:
The patent introduces dynamics by making the up-sampling process adaptive rather than static. The joint bilateral filter dynamically adjusts its spatial and range kernels based on local confidence values and depth gradients. This dynamic adaptation allows the system to maintain processing speed through efficient kernel selection while simultaneously improving depth data quality by preserving boundaries and reducing noise amplification in a localized manner.
Solution Approach 2:
The patent changes parameters dynamically during up-sampling by adjusting kernel sizes and filter strengths based on confidence map values and depth variations. High-confidence regions with strong depth gradients use smaller kernels to preserve boundaries, while low-variation regions use larger kernels for noise reduction. This parameter adaptation enables the system to achieve high processing speed while maintaining or improving depth data quality.
Data Source
AI summary
An image processing system includes a calculation unit, a reconstruction unit, a confidence map estimation unit and an up-sampling unit. The up-sampling unit is configured to perform a joint bilateral up-sampling on depth information of a first input image based on a confidence map of the first input image and a second input image with respect to an object and increase a first resolution of the first input image to a second resolution to provide an output image with the second resolution.