360-Degree Video PDU Prioritization for Spatially Adaptive Streaming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding and decoding technologies for immersive media, such as 360-degree and volumetric video, do not effectively utilize spatial adaptation of protocol data unit sets to optimize data transmission and decoding efficiency, particularly in wireless communication systems.
Innovation Solution
The method involves defining protocol data unit sets based on spatial regions within a frame of coded 360-degree video, assigning importance values to these sets based on spatial location and adaptation criteria, and adapting these values over time, along with signaling spatial adaptation of importance values to receivers for efficient decoding.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional video coding technologies are used for immersive media, then the transmission and decoding process is simpler, but the efficiency of data transmission and decoding is lower
Solution Approach 1:
The video frame is divided into multiple spatial regions, and each region is assigned a different importance value. This segmentation allows the system to prioritize transmission of critical regions while reducing bandwidth for less important areas, thereby improving transmission efficiency without requiring complete reprocessing of the entire frame.
Solution Approach 2:
Different importance values are assigned to different spatial regions within the same frame based on their significance to the user experience. This local differentiation enables optimized resource allocation where high-importance regions receive higher transmission priority and quality, while low-importance regions use fewer resources, improving overall productivity without uniform complexity increase.
2Quantity of substance
If spatial regions are not prioritized, then the transmission process is simpler, but the bandwidth requirements are higher
Solution Approach 1:
By segmenting the frame into spatial regions with different importance values, the system can allocate bandwidth dynamically to each region. High-importance regions receive sufficient bandwidth to maintain quality, while low-importance regions use reduced bandwidth, thereby reducing total bandwidth requirements without requiring complete reprocessing of the entire frame.
Solution Approach 2:
The system applies partial processing to different spatial regions based on their importance. Critical regions receive full processing and higher bandwidth allocation, while non-critical regions receive reduced processing and lower bandwidth allocation. This partial action approach reduces overall bandwidth consumption while maintaining acceptable quality for the most important content.
3Reliability
If uniform importance values are assigned to all PDU sets, then the processing is simpler, but the user experience in immersive media is degraded
Solution Approach 1:
Different importance values are assigned to different spatial regions based on their significance to user experience. This local quality differentiation ensures that regions critical to user experience (such as foveal vision areas in VR) receive higher priority and better quality transmission, while less critical regions use lower priority, thereby improving overall reliability of user experience without requiring complete reprocessing of the entire frame.
4Adaptability or versatility
If static importance values are used, then the processing is simpler, but the adaptation to changing spatial locations over time is poor
Solution Approach 1:
The importance values assigned to spatial regions are made dynamic rather than static. As the user's viewport or attention focus changes over time, the importance values are updated accordingly. This dynamic adaptation allows the system to automatically adjust to changing spatial locations and user preferences, improving adaptability without requiring complete reprocessing of the entire frame.
Solution Approach 2:
The system implements feedback mechanisms where information about user behavior, viewport position, or content importance is continuously monitored and used to adjust importance values in real-time. This feedback loop enables automatic adaptation to changing conditions, improving versatility while avoiding the need for manual reconfiguration or complete system reprocessing.
Data Source
AI summary
A sender of a bitstream of coded 360-degree video defines PDU sets of the coded 360-degree video. The PDU sets include spatial region(s) of a frame of the coded 360-degree video. The sender assigns protocol data unit set importances to the protocol data unit sets, wherein the assignment is based on a spatial location of the spatial region(s) included in the protocol data unit set and adaptation criteria. The sender adapts over time the PDU set importance values of the PDU sets corresponding to the spatial location based on the adaptation criteria. The sender sends, toward a receiver, the bitstream including the coded 360-degree video. A receiver receives the bitstream of coded 360-degree video, parses information, and outputs at least part of a decoded 360-degree video based on the parsed information.