3D Point Cloud Partitioning for Immersive Video Data Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current immersive video content technologies face challenges in efficiently rendering high-quality, free-viewpoint video due to the large amount of data required for rendering, especially for objects close to the cameras, leading to issues with parallax and data transport.
Innovation Solution
A method involving the partitioning of a three-dimensional point cloud into multiple parts, with each part having a depth map and color map determined based on two-dimensional parametrizations, represented in patch atlases, and a stream generated with mapping information to reduce data complexity and enhance rendering efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If free-viewpoint video techniques are used to represent and code multi-view video for immersive content, then user experience in immersive context is improved, but the amount of data to be transported to the renderer increases significantly
Solution Approach 1:
The point cloud representing the object is partitioned into a plurality of three-dimensional parts, each comprising at least a point of the three-dimensional point cloud. This segmentation allows the data to be organized into manageable patches that can be efficiently processed and transmitted, reducing the overall data transport burden while maintaining free-viewpoint capabilities
Solution Approach 2:
The patent transforms three-dimensional point cloud data into two-dimensional depth maps and color maps through dimensional projection. By representing 3D parts as 2D parametrizations with associated depth and color information, the system reduces data complexity while preserving the ability to render from arbitrary viewpoints, effectively trading one dimension for data efficiency
2Manufacturing precision
If immersive video content is obtained with mapping of video on a surface such as a sphere or a cube, then good image quality is provided, but issues related to parallax appear especially for objects in the foreground
Solution Approach 1:
The scene is divided into multiple three-dimensional parts or patches, each with its own depth map and color map. This segmentation allows foreground objects to be represented with appropriate geometric fidelity independent of the global spherical/cubic mapping, thereby reducing parallax artifacts while maintaining overall image quality
Solution Approach 2:
Different parts of the scene are processed with localized depth and color information tailored to their specific geometric characteristics. Foreground objects receive dedicated 3D part representations that preserve their true spatial relationships, while background areas can utilize the spherical/cubic mapping approach, optimizing both image quality and parallax accuracy locally
3Adaptability or versatility
If a three-dimensional point cloud is used to represent an object as seen from a range of points of view, then free-viewpoint rendering is enabled, but the data complexity and amount increase
Solution Approach 1:
The patent projects three-dimensional point cloud data onto two-dimensional surfaces to create depth maps and color maps. Each 3D part is represented by 2D parametrizations with associated depth values and color information, reducing data complexity while maintaining the capability to render from arbitrary viewpoints through the mapping information that preserves spatial relationships
4Manufacturing precision
If detailed representation of foreground objects is maintained for multiple viewpoints, then rendering quality is improved, but the amount of data to be transported increases
Solution Approach 1:
Foreground objects are segmented into distinct three-dimensional parts with dedicated depth maps and color maps. This segmentation allows detailed representation of only the necessary foreground elements rather than entire scenes, reducing data amount while maintaining rendering quality for important objects
Solution Approach 2:
The patent applies detailed 3D part representation selectively to foreground objects that require high rendering quality, while other areas can use less detailed representations. This partial application of full-detail processing optimizes the balance between rendering quality and data transmission requirements
Data Source
AI summary
Method and device for generating a stream of data representative of a 3D point cloud. The 3D point cloud is partitioned into a plurality of 3D elementary parts. A set of two-dimensional 2D parametrizations is determined, each 2D parametrization representing one 3D part of the point cloud with a set of parameters. Each 3D part is represented as a 2D pixel image. A depth map and a color map are determined as a first patch atlas and a second patch atlas. A data stream is generated by combining and/or coding the parameters of the 2D parametrization, the first patch atlas, the second patch atlas and mapping information that links each 2D parametrization with its associated depth map and color map in respectively the first and second patch atlas.