Occlusion-Based Point Cloud Density Reduction for Volumetric Video
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for determining occlusion in volumetric video streaming are computationally intensive and inefficient, particularly for high-quality videos, leading to scalability issues and high bandwidth consumption.
Innovation Solution
A heuristic algorithm is implemented on mobile devices to partition point clouds into cells, calculate occlusion levels based on predicted viewports, and reduce point density in occluded cells, allowing for efficient data transmission while maintaining visual quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If current occlusion determination methods are used for volumetric video streaming, then visual quality can be maintained, but computational complexity and bandwidth consumption increase significantly
Solution Approach 1:
The point cloud is divided into multiple cells, and occlusion determination is performed independently for each cell based on its position relative to the predicted viewport. This segmentation reduces the overall computational complexity by breaking down the global occlusion problem into multiple smaller local problems, while still maintaining accurate occlusion determination for each cell.
Solution Approach 2:
The patent applies different occlusion determination strategies to different cells based on their location. Cells closer to the viewport center undergo more rigorous occlusion checking, while peripheral cells use simplified determination. This local quality approach maintains high accuracy for critical regions while reducing computational burden for less important areas.
2Measurement precision
If current occlusion determination methods are used for volumetric video streaming, then visual quality can be maintained, but bandwidth consumption increases
Solution Approach 1:
The patent extracts and transmits only the visible points from each cell that pass the occlusion determination test, rather than transmitting all points. By taking out only the necessary visible information and discarding occluded points, the bandwidth consumption is significantly reduced while maintaining the accuracy of the visible portions.
Solution Approach 2:
The patent performs occlusion determination at a selective level - not all cells require full occlusion checking. By applying partial action (simplified occlusion determination) to cells that are less likely to be visible or less critical, the system reduces the overall processing load and associated bandwidth requirements while maintaining sufficient accuracy for the final rendered view.
3Quantity of substance
If point density is reduced in occluded cells, then bandwidth consumption decreases, but visual quality may be compromised
Solution Approach 1:
The patent converts the potential harm of reduced point density into a benefit by using occlusion determination as a filter. Points that would be occluded anyway are identified and excluded, so reducing their density does not harm visual quality. The 'harm' of lower resolution in occluded areas becomes beneficial because those areas contribute nothing to the final visible image, thus saving bandwidth without sacrificing quality.
4Productivity
If heuristic algorithm is implemented on mobile devices, then processing speed improves, but device energy consumption increases
Solution Approach 1:
The patent performs occlusion determination as a preliminary filtering step before point transmission and rendering. By pre-identifying and eliminating occluded points on the mobile device, the system avoids the energy-intensive task of transmitting and then discarding unnecessary points later. This preliminary action reduces the overall computational workload and energy consumption during the main rendering process.
Data Source
AI summary
Aspects of the subject disclosure may include, for example, a device, that includes a processing system including a processor and a memory that stores executable instructions that, when executed by the processing system, facilitate performance of operations including receiving a manifest for a point cloud, wherein the point cloud is partitioned into a plurality of cells; determining an occlusion level for a cell of the plurality of cells with respect to a predicted viewport; reducing a point density for the cell provided in the manifest based on the occlusion level, thereby determining a reduced point density; and requesting delivery of points in the cell, based on the reduced point density. Other embodiments are disclosed.