VR Depth Remapping with Real-World Bounds for HEVC Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing VR depth maps suffer from lossy compression issues, leading to inaccurate re-creation of optical depths due to insufficient resolution and the limitations of HEVC, especially in areas with depth discontinuities and near-black pixels, which results in blocky and uneven object surfaces.
Innovation Solution
Re-map VR depth map optical depths based on corresponding real-world depth map bounds to reduce the number of bits required for encoding while maintaining accuracy, using predefined values to determine lower and upper bounds for each pixel, allowing for more precise decoding and occlusion determination.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If lossy video compression (HEVC) is used to reduce network traffic, then network efficiency is improved, but depth map accuracy deteriorates due to compression artifacts and insufficient resolution
Solution Approach 1:
The patent applies parameter changes by transforming the optical depth values from a large dynamic range (0 to infinity or 0 to maximum depth) into a normalized range (0 to 1) through division by a scale factor. This parameter transformation allows the depth map to be efficiently compressed using HEVC while preserving sufficient accuracy for occlusion testing, as the normalized values maintain the relative depth relationships needed for correct rendering.
Solution Approach 2:
The patent performs preliminary action by normalizing the optical depth values before compression. The server divides each optical depth value by a scale factor (e.g., 1.0, 0.5, or 0.25) to normalize the depth range to (0, 1), and transmits this normalized depth map to the client. This preliminary normalization ensures that the subsequent lossy compression does not introduce excessive artifacts, as the values are already scaled to a manageable range.
2Ease of manufacture
If optical depth range is remapped to 0-1 for HEVC encoding, then encoding compatibility is improved, but effective resolution deteriorates to approximately 1.9cm or 0.5cm
Solution Approach 1:
The patent uses parameter changes by introducing a scale factor that can be dynamically adjusted. Instead of a fixed remapping to 0-1, the system divides optical depths by a scale factor (1.0, 0.5, or 0.25) to create normalized values in the range (0, 1). This flexible parameter adjustment allows the system to maintain higher effective resolution (better than 0.5cm) while still being compatible with HEVC encoding requirements.
Solution Approach 2:
The patent applies dynamics by making the scale factor adjustable rather than fixed. The server can select different scale factors (1.0, 0.5, or 0.25) based on the specific depth range and compression needs. This dynamic adjustment allows the system to optimize the balance between encoding compatibility and depth resolution for different scenarios, rather than being constrained by a single fixed remapping scheme.
3Measurement precision
If fixed-size tiles are used for depth remapping, then accuracy in uniform depth regions is improved, but complexity increases due to additional processing and data transmission requirements
Solution Approach 1:
The patent applies segmentation by dividing the depth map into multiple tiles with different scale factors. Instead of using a single uniform remapping for the entire image, the system segments the depth map into regions (tiles) and applies appropriate scale factors to each region based on its depth characteristics. This allows higher accuracy in uniform depth regions while avoiding the complexity of processing every pixel individually.
Solution Approach 2:
The patent uses local quality by applying different scale factors to different regions of the depth map based on their specific depth characteristics. Rather than applying a single global remapping, the system identifies regions with similar depth ranges and applies localized remapping strategies. This allows optimization of depth accuracy for each region while reducing overall processing complexity compared to per-pixel analysis.
Data Source
AI summary
A real-world depth map is obtained corresponding to a viewpoint from a perspective of which a virtual-reality (VR) depth map has been generated. For a given pixel in the VR depth map, an optical depth (D) of a corresponding pixel in the real-world depth map is found. A lower bound (D−D1) for the given pixel is determined by subtracting a first predefined value (D1) from the optical depth (D) of the corresponding pixel in the real-world depth map. An upper bound (D+D2) for the given pixel is determined by adding a second predefined value (D2) to the optical depth (D) of the corresponding pixel in the real-world depth map. An optical depth of the given pixel fetched from the VR depth image is re-mapped, from a scale of the lower bound to the upper bound determined for the given pixel to another scale of A to B, wherein A and B are scalars. Re-mapped optical depths of pixels of the VR depth map are then encoded into an encoded depth map. This encoded depth map is sent to at least one display apparatus.