Depth Map Segmentation Using Depth Ranges for Low-Bitrate Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multi-view imaging technologies face challenges in efficiently transmitting and rendering depth maps due to high computational demands, irregular patch shapes, and limited support for alpha channels, which affect spatial and temporal consistency and bitrate requirements, particularly in real-time telepresence applications.
Innovation Solution
The method involves identifying depth regions and determining ranges of depth values for each region, transmitting these ranges instead of segmentation masks, and using them to segment and render multi-view frames with reduced computational complexity and bitrate.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If segmentation masks are transmitted to provide segmentation information, then segmentation accuracy is improved, but bitrate requirement increases
Solution Approach 1:
The patent extracts only the essential segmentation information (depth ranges) from the full segmentation mask, transmitting minimal data while preserving the ability to perform accurate segmentation at the decoder side. This reduces bitrate while maintaining segmentation accuracy.
Solution Approach 2:
Instead of transmitting the actual segmentation mask, the patent transmits depth range parameters that enable the decoder to reconstruct or infer the segmentation information. This copying approach uses significantly less data while achieving the same functional result.
2Adaptability or versatility
If irregular patch shapes are placed in atlas, then segmentation flexibility is improved, but spatial and temporal consistency deteriorates
Solution Approach 1:
The patent segments the depth map into regions defined by depth ranges, allowing flexible object separation without requiring irregular patch boundaries in the atlas. This maintains consistency while preserving segmentation flexibility through parameter-based region definition.
Solution Approach 2:
The patent changes from spatial-based patch definitions to parameter-based depth range definitions. This allows the same atlas structure to handle flexible segmentation by varying depth range parameters rather than reconfiguring patch geometries, maintaining spatial and temporal consistency.
3Measurement precision
If per-frame variable number of patches are used, then segmentation precision is improved, but computational complexity increases
Solution Approach 1:
The patent performs preliminary depth range determination at the encoder side, organizing segmentation information in advance as depth range parameters. This preliminary organization reduces the computational burden at the decoder, maintaining segmentation precision while lowering real-time processing complexity.
4Ease of operation
If full view textures and full view images are directly placed into image frame, then transmission simplicity is improved, but total number of views that can be transmitted deteriorates
Solution Approach 1:
The patent segments the scene into depth-based regions and transmits only the necessary depth map and range information rather than complete full-view images for each view. This segmentation approach reduces per-view data requirements, enabling more views to be transmitted within the same bandwidth while maintaining simplicity through standardized depth map transmission.
Data Source
AI summary
A method is provided for processing a depth map for subsequent transmission. Depth regions in the depth map are identified, each of the depth regions defining a portion of the depth map. Ranges of depth values are determined for the depth regions and are further transmitted with the depth map. The ranges of depth values provide segmentation information of the depth regions in the depth map.