Auxiliary Information Map Upsampling Using Texture Guidance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current depth and disparity map upsampling methods, such as nearest neighbor, linear, bilinear, or bicubic interpolation, introduce incorrect distance information and cause smearing effects, especially around object boundaries, leading to poor quality virtual views in 3D media applications.
Innovation Solution
A method for upsampling auxiliary information maps, such as depth or disparity maps, by selecting reference pixels based on texel values from an associated texture, calculating updated pixel values using median or average methods, and optionally applying smoothing filters to improve accuracy and reduce artifacts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If standard image or video upsampling methods (nearest neighbor, linear, bilinear, or bicubic interpolation) are applied to depth maps, then the resolution is increased, but incorrect distance information is introduced causing smearing effects around object boundaries
Solution Approach 1:
The patent applies different upsampling strategies to different regions of the depth map based on local characteristics. Texture information from the associated texture map is used to identify regions with object boundaries versus smooth regions, allowing the algorithm to selectively apply appropriate filtering methods that preserve edge sharpness while maintaining overall resolution improvement.
Solution Approach 2:
The patent introduces texture information from an associated texture map as an intermediary to guide the upsampling process. This texture data serves as additional information that helps distinguish between different regions (object boundaries vs. smooth areas), enabling more accurate depth value interpolation that avoids smearing effects.
2Loss of energy
If depth maps are transmitted with reduced resolution to reduce bitrate, then transmission cost is reduced, but quality of virtual view synthesis deteriorates
Solution Approach 1:
The patent performs high-quality upsampling of the depth map using texture-guided methods before virtual view synthesis. This preliminary high-quality processing compensates for the reduced resolution transmission, ensuring that the final synthesized views maintain high quality despite the lower bitrate transmission of the original depth map.
3Device complexity
If conventional upsampling filters are used on depth maps, then computational simplicity is maintained, but smearing artifacts are introduced around object boundaries
Solution Approach 1:
The patent modifies the upsampling process to treat different regions of the depth map differently based on local texture characteristics. By using texture information to identify object boundaries, the algorithm applies edge-preserving filtering in boundary regions while using standard filtering in smooth regions, thereby reducing smearing artifacts without requiring complex global processing.
Solution Approach 2:
The patent combines multiple information sources (depth map pixels, texture map information, and gradient calculations) to create a composite upsampling approach. This composite method leverages the strengths of different data types to achieve artifact reduction while maintaining computational efficiency.
Data Source
AI summary
An auxiliary information map (10) is upsampled to form an upsampled auxiliary information map (20). Multiple reference pixels (23) in the upsampled auxiliary information map (20) are selected for a current pixel (21) in the upsampled auxiliary information map (20) based on texel values of texels in an associated texture (30). An updated pixel value is calculated for the current pixel (21) based on the pixel values of the selected reference pixels (23).