Viewport-Dependent Content Delivery Using Consumption Heat Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Virtual reality and 360° content delivery faces challenges due to bandwidth constraints, resulting in high latency and choppy playback, as existing methods struggle to efficiently transmit high-resolution content while ensuring seamless user experience.
Innovation Solution
The proposed solution involves viewport-dependent delivery methods, where content is encoded, pre-fetched, and cached based on consumption data, with frequently viewed areas receiving higher quality settings and priority, and predictive data being used for unviewed content, utilizing heat maps and machine learning to optimize encoding, caching, and rendering processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all content is transmitted at high resolution to provide realistic virtual reality effects, then the quality of user experience is improved, but bandwidth consumption increases causing high latency and choppy playback
Solution Approach 1:
The patent applies local quality by encoding different spatial portions of the content at different quality levels based on consumption data. Frequently viewed portions (identified through heat maps) are encoded with higher quality settings and bitrates, while infrequently viewed portions use lower quality settings. This resolves the contradiction by maintaining high user experience quality for viewed areas while reducing overall bandwidth consumption.
Solution Approach 2:
The patent segments the content into multiple spatial portions and processes each segment independently with different encoding parameters. By dividing the content based on consumption patterns, the system can allocate bandwidth efficiently to different segments, ensuring high quality for important areas while reducing total bandwidth usage.
2Reliability
If high resolution content is delivered to maintain immersion, then the realism of virtual reality is improved, but network performance deteriorates due to bandwidth constraints
Solution Approach 1:
The system maintains immersion quality in frequently viewed spatial portions by applying higher encoding quality settings to those specific areas, while reducing quality in less important areas. This local differentiation preserves the immersive experience where users actually look while improving overall delivery speed through reduced total data transmission.
3Loss of energy
If viewport dependent delivery is used to transmit only visible portions, then bandwidth efficiency is improved, but system complexity increases due to tracking and selective encoding requirements
Solution Approach 1:
The system performs preliminary action by pre-processing content into multiple spatial portions with different quality levels before delivery, based on consumption data and heat maps. This pre-segmentation and pre-encoding approach simplifies the real-time delivery process, as the complex segmentation and quality assignment is done in advance rather than during active streaming.
Solution Approach 2:
The system creates multiple versions (copies) of the content with different quality settings for different spatial portions. These pre-prepared copies are then delivered based on user viewport and consumption patterns, avoiding the need for complex real-time encoding decisions during streaming.
4Reliability
If quality settings are increased for frequently viewed portions to improve user experience, then viewing quality is improved, but overall data transmission volume increases
Solution Approach 1:
The patent implements local quality by applying different quality settings to different spatial portions based on consumption data. Frequently viewed portions receive higher quality encoding while infrequently viewed portions use lower quality settings. This ensures high viewing quality where needed while controlling overall data transmission volume through selective quality allocation.
Data Source
AI summary
A method, apparatus and computer program product are provided for improving the efficiency of content delivery based on consumption data. Consumption data may be collected from various users and/or devices describing frequently viewed areas in the content. The consumption data may be represented as a heat map. Variable encoding properties, caching priorities, and/or rendering priorities may be determined based on the consumption data. Content encoding and delivery may therefore be optimized accordingly. Machine learning may be applied to create predicted consumption data for content which lacks actual consumption data, allowing optimizations to be applied in advance of any actual consumption. Some concepts may be applied to unviewed video to optimize live streaming or the like.