Volumetric Video Rendering via Point Cloud Tile Sorting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Volumetric video rendering faces challenges due to the large amount of data required for 3D content, which leads to bandwidth and processing issues, especially when capturing with multiple cameras, and existing methods struggle with efficient surface reconstruction and rendering, particularly in screen space, causing performance problems and unrealistic representations.
Innovation Solution
The method involves processing input point cloud tile data by arranging it into bounding boxes, sorting and associating these boxes with rendering tiles, storing overlapping indices, and reconstructing screen space surfaces using these indices to optimize rendering and reduce bandwidth and processing demands.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of moving object
If volumetric 3D video content is captured using multiple 3D cameras, then the field of view and immersion experience are improved, but the data quantity and bandwidth requirements increase significantly
Solution Approach 1:
The patent divides the volumetric scene into multiple 3D tiles, each containing a subset of point cloud data. This segmentation allows the system to process and transmit only the necessary portions of data for each tile rather than the entire volumetric scene, significantly reducing bandwidth requirements while maintaining the ability to render immersive 3D content with wide field of view.
2Reliability
If all point cloud data is rendered to the screen space surface, then the representation completeness is improved, but the rendering time and processing bandwidth increase
Solution Approach 1:
The patent applies different rendering strategies to different tiles based on their importance and visual quality requirements. Critical tiles with significant visual content are rendered with higher fidelity and more processing resources, while less important tiles are rendered with reduced detail or skipped entirely. This local quality approach maintains representation completeness for essential areas while dramatically reducing overall rendering time and processing bandwidth.
3Productivity
If bounding boxes are sorted and associated with rendering tiles, then the rendering efficiency is improved, but the algorithm complexity increases
Solution Approach 1:
The patent performs preliminary sorting and association of bounding boxes with rendering tiles before the actual rendering process. By pre-organizing the data structure to map bounding boxes to their corresponding tiles in advance, the system eliminates the need for complex real-time calculations during rendering, improving rendering efficiency while keeping the added algorithmic complexity manageable through systematic pre-processing.
Data Source
Figure 1
Figure 2a~2b
Figure 2c
AI summary
The invention relates to a method comprising processing a data stream comprising input point cloud tile data arranged into bounding boxes (BBs); sorting the BBs according to a viewing location; dividing a rendering framebuffer into rendering tiles; associating BBs with the rendering tiles; storing indices of BBs that are overlapping in overlapped rendering tiles; reconstructing a screen space surface by rendering the points of the BBs to rendering screen space tile points, by utilizing the stored indices for the overlapped rendering tiles; and rendering the reconstructed screen space.