Volumetric Video Patch Mapping for Real-Time Streaming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current volumetric video coding technologies face challenges in efficiently compressing and decompressing 3D scene data for AR, VR, and MR applications, particularly in handling large datasets and maintaining quality during transmission and storage.
Innovation Solution
The proposed solution involves a compression process that decomposes point cloud data into patches, generates geometry and texture images, and uses occupancy maps to minimize unused space, with auxiliary information encoded for efficient decoding and reconstruction of 3D point clouds, leveraging ISO/IEC 23090-5 standards for volumetric video coding.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If volumetric video data is compressed using traditional coding technologies, then storage and transmission requirements are reduced, but compression efficiency and quality maintenance deteriorate
Solution Approach 1:
The volumetric video data is divided into multiple patches from the point cloud, and each patch is processed independently to generate corresponding geometry images and texture images. This segmentation allows efficient compression of smaller data units while maintaining overall quality, resolving the contradiction between reducing data quantity and maintaining compression efficiency.
Solution Approach 2:
The patent transforms 3D point cloud data into 2D geometry images and texture images through projection. This dimensional transformation enables the use ofๆ็ 2D video compression techniques to handle volumetric data, significantly improving compression efficiency while reducing storage and transmission requirements.
2Ease of operation
If volumetric video data is decompressed and rendered in real-time, then AR/VR/MR applications can function seamlessly, but processing time and computational resources increase
Solution Approach 1:
The encoder pre-processes the point cloud data by decomposing it into patches and generating corresponding geometry and texture images before transmission. This preliminary action reduces the computational burden during real-time rendering at the decoder, enabling seamless AR/VR/MR applications while minimizing processing time delays.
Solution Approach 2:
The patent creates 2D copies (geometry images and texture images) from the 3D point cloud data that can be efficiently decoded and rendered. These copies serve as intermediate representations that accelerate the real-time rendering process, improving ease of operation in AR/VR/MR applications without significant time loss.
3Productivity
If point cloud data is decomposed into patches and processed independently, then compression efficiency improves, but mapping accuracy between patches and original data deteriorates
Solution Approach 1:
The patent incorporates feedback mechanisms where the decoder receives mapping information about patch correspondences and uses this to accurately reconstruct the 3D point cloud. The feedback loop ensures that compression efficiency gains from independent patch processing do not compromise mapping accuracy, as the system continuously refines the reconstruction based on received mapping data.
Solution Approach 2:
The patent introduces mapping information as an intermediary element that connects the decomposed patches to the original point cloud data. This intermediary layer preserves the relationship between patches and source data, ensuring accurate reconstruction while maintaining the compression efficiency benefits of independent patch processing.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The embodiments relate to a method comprising obtaining at least two bitstreams each comprising one or more independently decodable sub-bitstreams; determining identifiers of the one or more independently decodable sub-bitstream and a spatial relationship between the independently decodable sub-bitstreams of the at least two bitstreams; generating a description file, in which an indication of the relationship between the sub-bitstream identifiers of the independently decodable sub-bitstreams from the at least two bitstreams is included; encapsulating the at least two bitstreams into real-time protocol streams; and transmitting the real-time protocol packets comprising the at least two real-time protocol streams along with the description file indicating the relations between the independently decodable sub-bitstreams of the at least two bitstreams. There is also described a method comprising obtaining a description file comprising at least one mapping table, and two or more RTP media stream descriptions; determining a relationship between independently decodable sub-bitstreams by parsing the mapping table; requesting from a sender a portion of the two or more bitstreams; and receiving the requested portion of the two or more bitstreams.