MMTP Signaling for Viewport-Dependent Point Cloud Streaming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies face challenges in efficiently streaming and compressing visual volumetric video-based coding (V3C) and geometry-based point cloud coding (G-PCC) media, particularly in supporting lossy and lossless coding of point cloud geometry coordinates and attributes, and do not provide adequate signaling mechanisms for dynamic adaptive streaming over HTTP.
Innovation Solution
The implementation of methods and systems for streaming V3C media and G-PCC media using the MPEG Media Transport Protocol (MMTP), which includes defining new signaling elements and encapsulation techniques to enable view-dependent streaming and efficient delivery of point cloud data, allowing clients to select and retrieve only the necessary assets based on the viewport and network constraints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If all media assets are streamed to support complete point cloud data, then data completeness is improved, but bandwidth consumption increases
Solution Approach 1:
The point cloud data is divided into multiple media assets representing different regions or resolutions. The sending device segments the complete point cloud into manageable chunks that can be selectively streamed based on client needs, allowing partial data transmission while maintaining data completeness for the required viewport region.
Solution Approach 2:
Different regions of the point cloud are encoded with different qualities based on their importance to the current viewport. High-priority regions within the visible viewport are streamed with higher quality, while regions outside the viewport or less important areas are omitted or streamed at lower quality, optimizing bandwidth usage while preserving essential visual information.
2Manufacturing precision
If complete point cloud data is transmitted, then rendering quality is improved, but transmission time increases
Solution Approach 1:
The sending device performs preliminary analysis to identify which portions of the point cloud data are necessary for the current viewport before transmission. Media assets are pre-prepared and prioritized based on their relevance to the expected or current viewing region, allowing the client to quickly receive and render essential data without waiting for complete data transmission.
Solution Approach 2:
Instead of transmitting the complete point cloud dataset, the system transmits only the partial data necessary for the current viewport. This partial action approach provides sufficient rendering quality for the visible region while dramatically reducing transmission time compared to sending all available data.
3Loss of energy
If adaptive streaming is implemented to reduce bandwidth usage, then bandwidth efficiency is improved, but system complexity increases
Solution Approach 1:
The system implements feedback mechanisms where the sending device receives information about the client's current viewport, device capabilities, and network conditions. Based on this feedback, the sending device dynamically adjusts which media assets to stream and at what quality levels, achieving bandwidth efficiency through adaptive decision-making without requiring complex client-side processing.
Solution Approach 2:
The patent introduces signaling protocols and metadata structures that act as intermediaries between the sending and receiving devices. These intermediaries carry information about available media assets, their characteristics, and selection criteria, enabling adaptive streaming through standardized communication rather than requiring complex custom implementations on either end.
4Loss of energy
If view-dependent streaming is implemented to stream only necessary data, then bandwidth consumption is reduced, but signaling complexity increases
Solution Approach 1:
The signaling protocol is designed to be multi-functional, serving both as a capability advertisement mechanism and as a viewport-dependent asset selection mechanism. The same message structures convey information about available media assets, their spatial relationships, and selection criteria, eliminating the need for separate signaling channels and reducing overall signaling complexity.
Data Source
AI summary
Methods, systems, and apparatuses for streaming of visual volumetric video-based coding (V3C) media and geometry-based point cloud coding (G-PCC) media are described herein. A method implemented in a receiving device may include receiving one or more of a first message including a list of media assets that are available to be streamed from the sending device, or one or more messages respectively describing the media assets. The method may further include sending a second message indicating a request for a subset of the media assets to be streamed from the sending device. The requested subset of the media assets may be determined based on a viewport of the receiving device. The method may further include receiving Motion Picture Experts Group (MPEG) Media Transport Protocol (MMTP) packets and processing the packets to recover at least a portion of the requested subset of the media assets.