SphericRTC Content-Adaptive 360-Degree Video Transmission
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Real-time 360-degree video communication faces challenges in bandwidth efficiency and visual quality due to the transmission of unviewed pixels, particularly in immersive experiences like teleconferencing and telepresence, where current methods fail to accurately predict user view orientations quickly enough to adapt frame representations effectively.
Innovation Solution
SphericRTC employs content-adaptive oriented projections, offloading transformations to a GPU for real-time processing, using recent user orientations to prioritize transmission of viewed pixels over unviewed ones, thereby reducing bandwidth consumption and enhancing view quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If full omnidirectional frames are transmitted, then complete 360-degree content is delivered, but bandwidth is wasted on unviewed pixels
Solution Approach 1:
The patent extracts only the necessary portion of the omnidirectional frame (the viewport region) for transmission. By identifying and transmitting only the pixels within the user's field of view rather than the complete 360-degree frame, the system eliminates wasted bandwidth while maintaining complete visual information for the user's actual viewing area.
Solution Approach 2:
The patent applies different quality levels to different regions of the omnidirectional frame. High-quality encoding is applied specifically to the viewport region where the user is looking, while lower quality or no transmission is applied to regions outside the viewport. This local differentiation optimizes bandwidth usage by concentrating resources on the visually important area.
2Loss of energy
If tiling or oriented projection approaches are used to reduce unviewed pixels, then bandwidth efficiency improves, but accurate prediction of user view orientation is required
Solution Approach 1:
The patent implements a feedback mechanism where the transmitted viewport information is sent back to the sender, enabling continuous refinement of view orientation predictions. This feedback loop allows the system to adapt to actual user viewing behavior and improve prediction accuracy over time, resolving the contradiction between bandwidth efficiency and prediction accuracy requirements.
Solution Approach 2:
The patent performs preliminary transformation of the omnidirectional frame into tile-based or oriented projection representations before transmission. By pre-processing the content into formats optimized for specific viewport orientations, the system reduces the computational burden during real-time viewing and enables faster adaptation when view orientation changes, thereby improving both bandwidth efficiency and response to prediction accuracy.
3Loss of energy
If real-time transformation to tile-based or oriented projection segments is performed, then bandwidth efficiency improves, but processing time is reduced
Solution Approach 1:
The patent segments the omnidirectional frame into multiple tiles or projection segments that can be independently processed and transmitted. This segmentation allows parallel processing of different regions, reducing overall processing time while maintaining the bandwidth efficiency benefits of selective transmission. Only the relevant viewport tiles are transmitted in real-time, eliminating the need to process and transmit the entire frame.
Solution Approach 2:
The patent applies partial transformation by processing only the necessary portion of the omnidirectional frame (the viewport region) rather than transforming the entire 360-degree content. This partial action approach reduces processing time significantly while still achieving bandwidth efficiency by avoiding transmission of unviewed pixels outside the viewport.
4Adaptability or versatility
If new content is generated on-the-fly in real-time communication, then interactivity is enabled, but traces of view orientations from many users cannot be used to predict current user's view
Solution Approach 1:
The patent implements self-service by having each user device independently determine and communicate their own viewport information back to the sender. Rather than relying on predictions from other users' viewing patterns, each user's viewport is self-reported, enabling real-time interactivity with new content generation while maintaining accurate knowledge of each user's actual view orientation without requiring predictive algorithms.
Data Source
AI summary
SphericRTC provides real-time 360-degree video communication, which allows the viewer to observe the environment in any direction from the camera location. This more allows users to more-efficiently exchange information and can be beneficial in the real-time setting. The system selects representations of 360-degree frames to allow efficient, content-adaptive delivery. The system performs joint content and bitrate adaptation in real-time by offloading expensive transformation operations to a GPU. The system demonstrates that the multiple sub-components: viewport feedback, representation selection, and joint content and bitrate adaptation, can be effectively integrated within a single framework. Compared to a baseline implementation, views in SphericRTC have consistently higher visual quality. The median Viewport-PSNR of such views is 2.25 dB higher than views in the baseline system.