Video-Based Point Cloud Compression with Additional Missed-Point Patches
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies face challenges in efficiently compressing and decompressing point clouds for immersive media applications, leading to significant bandwidth requirements and artifacts like cracks and holes in reconstructed point clouds due to the inability to effectively encode and decode 3D video content.
Innovation Solution
The system generates additional points patches to represent missed points in 3D point clouds, which are then encoded and included in 2D frames, allowing for efficient compression and transmission using existing video codecs, thereby improving the visual quality of reconstructed point clouds.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If point clouds are compressed prior to transmission to reduce bandwidth requirements, then transmission efficiency is improved, but visual quality of reconstructed point clouds deteriorates due to artifacts like cracks and holes
Solution Approach 1:
The point cloud data is segmented into occupied and unoccupied regions using occupancy maps. The encoding process separately handles occupied patches (with geometry and texture data) and unoccupied regions (with placeholder values), allowing efficient compression while preserving visual quality by focusing bandwidth on important occupied regions.
Solution Approach 2:
Occupancy maps serve as an intermediary structure that guides the encoding and decoding process. These maps indicate which regions contain actual point cloud data and which are empty, enabling the decoder to reconstruct the point cloud accurately by placing points only in indicated occupied regions, thereby reducing artifacts.
2Device complexity
If conventional video codecs are used to compress point cloud data, then device complexity is reduced, but the ability to accurately represent 3D spatial relationships deteriorates
Solution Approach 1:
The patent transforms 3D point cloud data into a 2D patch-based representation that can be processed by conventional video codecs. By projecting 3D points onto 2D patches and using occupancy maps to indicate spatial occupancy, the system enables standard video compression to handle 3D data while preserving spatial relationships through the patch structure and occupancy information.
3Measurement precision
If all points in the point cloud are encoded to maintain complete information, then measurement precision is improved, but bandwidth requirements increase significantly
Solution Approach 1:
Instead of encoding all points uniformly, the patent applies partial encoding by using occupancy maps to identify and encode only occupied regions. Unoccupied regions are represented with placeholder values rather than full point data, achieving a balance between data completeness and bandwidth efficiency by encoding only the necessary portions.
Data Source
AI summary
A decoding device, an encoding device and a method for point cloud decoding is disclosed. The method includes generating, for a 3D point cloud, a first 2D frame representing a first attribute and a second 2D frame representing a second attribute. The first 2D frame and the second 2D frame include respective clusters of projected points from the 3D point cloud. The method includes detecting missed points of the 3D point cloud and generating first and second additional points patches representing the first attribute and the second attribute, respectively, based on at least a subset of the missed points. The method includes including the first and second additional points patch in the first and second 2D frame, respectively. The method includes encoding the first 2D frame and the second 2D frame to generate a compressed bitstream and transmitting the compressed bitstream.