Foveated Rendering VR Video Streaming Bandwidth Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current 360° virtual reality video streaming technologies face challenges in cost-effective skip-free delivery, particularly on mobile devices, due to hardware performance and software compatibility issues, and lack effective methods for analyzing user feedback and adapting content in real-time based on user profiles.
Innovation Solution
A system comprising a source of video content, a transmitter, a receiver, a feedback unit, and a processor that analyzes and predicts user journey data to modify and adapt video content in real-time, using foveated rendering to transmit only high-watchability content based on user profiles and behavior, reducing network and GPU load.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If foveated rendering is used to transmit only high-watchability content based on user profiles, then network bandwidth and GPU load are reduced, but video quality in peripheral regions deteriorates
Solution Approach 1:
The patent applies foveated rendering to transmit video content with varying quality levels in different spatial regions. The central foveal region receives high-resolution content while peripheral regions receive lower-resolution content, optimizing network bandwidth usage while maintaining perceived video quality where users actually look.
Solution Approach 2:
The system dynamically changes video encoding parameters (resolution, bitrate) based on the user's gaze position and head orientation. The encoder adjusts the quality parameters of different video regions in real-time according to the predicted foveal area, achieving adaptive quality optimization.
2Adaptability or versatility
If user feedback is collected and analyzed in real-time to adapt content, then user engagement improves, but system complexity and processing requirements increase
Solution Approach 1:
The patent implements a feedback loop where user interactions (gaze position, head movements, viewing duration) are continuously collected, analyzed, and used to adjust content delivery. The system processes user feedback in real-time to update user profiles and modify video encoding parameters dynamically.
Solution Approach 2:
The system pre-processes and stores user profile data and content metadata before actual video delivery. User profiles are built incrementally from historical data, and content is pre-analyzed for key features, enabling faster real-time adaptation without excessive processing complexity during streaming.
3Speed
If low-resolution images are initially sent and updated with higher resolution portions, then initial loading time is reduced, but total transmission time increases
Solution Approach 1:
The system sends a low-resolution placeholder or keyframes immediately to enable rapid initial display, then progressively transmits higher-resolution portions based on predicted user gaze and actual viewing behavior. This preliminary action ensures immediate visual feedback while optimizing subsequent data transmission.
Solution Approach 2:
The video transmission transitions from static resolution to dynamic resolution adjustment. The system continuously adapts the transmitted resolution based on real-time user gaze tracking and head orientation, sending higher resolution only to currently viewed regions while maintaining lower resolution in non-viewed areas.
4Manufacturing precision
If 360° virtual reality video is delivered in full resolution, then video quality is maximized, but network bandwidth consumption and device performance requirements increase significantly
Solution Approach 1:
The patent applies foveated rendering to transmit video content with varying quality levels in different spatial regions. The central foveal region receives high-resolution content while peripheral regions receive lower-resolution content, optimizing network bandwidth usage while maintaining perceived video quality where users actually look.
Solution Approach 2:
The 360° video content is segmented into multiple resolution zones based on spherical coordinates and user gaze direction. The encoding process divides the video sphere into foveal, intermediate, and peripheral regions, applying different quality levels to each segment to reduce overall bandwidth requirements.
Data Source
AI summary
A device and methods are disclosed for delivering video content to an end user comprising: a. a source of video content comprising a sequence of scenes; b. a transmitter for transmitting said video content; c. a receiver for receiving the video content and displaying the video content to the end user; d. a feedback unit for sensing a user's journey; e. a processor preprogrammed to modify the video content to be transmitted according to sensed user's journey; wherein the processor is configured to analyze sensed user's activity and predict user's journey in future such that a portion of video content which has highest predicted watchability by the user is transmitted and displayed.