Eye-Gaze Adaptive Multi-View Streaming for Bandwidth Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current multi-view video streaming applications struggle to dynamically adjust to users' shifting attention, limiting the number of views and perspectives due to technical constraints and user attention limitations, failing to replicate the natural way attention shifts during in-person events.
Innovation Solution
Leveraging eye-gaze tracking with inexpensive cameras and machine learning to dynamically adjust screen size and resolution of multiple views based on user attention, using adaptive bit-rate encoding for seamless transitions and backup streams to support user focus changes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multi-view video streaming applications display multiple views simultaneously, then user perspective options increase, but device bandwidth and processing resources are overwhelmed
Solution Approach 1:
The system applies different quality levels (resolution, bitrate) to different video views based on their importance and user attention. The focused view receives high quality encoding while peripheral views use lower quality, optimizing bandwidth usage while maintaining user experience.
Solution Approach 2:
The system dynamically changes encoding parameters (bitrate, resolution, frame rate) based on user eye gaze data and view importance. When a view becomes the focus, its parameters are increased; when peripheral, parameters are reduced, allowing flexible adaptation to user needs without constant high bandwidth consumption.
2Manufacturing precision
If the system encodes video at high bit rates for all views, then video quality is maintained, but network bandwidth consumption increases
Solution Approach 1:
Different quality levels are applied to different views based on their current importance. The focused view maintains high video quality with high bitrate encoding, while peripheral views use lower bitrate encoding, reducing overall bandwidth consumption while preserving quality where needed.
Solution Approach 2:
The system applies high-quality encoding only partially - specifically to the views that users are currently attending to - rather than encoding all views at maximum quality. This partial application of high-quality encoding significantly reduces bandwidth consumption while maintaining user experience.
3Adaptability or versatility
If the system switches between different bit rate streams, then adaptive quality is achieved, but decoding complexity and buffer requirements increase
Solution Approach 1:
The system performs encoding at multiple bit rates in advance during the encoding phase, creating a scalable bitstream structure. This preliminary preparation allows the decoder to simply select and decode the appropriate quality level without complex real-time encoding decisions, reducing decoding complexity while maintaining adaptive quality.
4Manufacturing precision
If multiple high-resolution streams are transmitted simultaneously, then view quality is maintained, but network latency increases
Solution Approach 1:
The system transmits high-resolution data only for the focused view that users are currently attending to, while using lower resolution for peripheral views. This selective high-quality transmission reduces overall data volume and network latency while maintaining perceived view quality.
Solution Approach 2:
Instead of transmitting full high-resolution data for all views simultaneously, the system partially transmits high-quality data only for the currently focused view, reducing total bandwidth requirements and network latency while maintaining quality where users are actually looking.
Data Source
AI summary
A method uses the eye gaze of a user, as a proxy of their attention, and leverages it to provide a more natural experience in a multi-view, i.e. multi-party and multi-perspective, video streaming service. The method takes advantage of increasingly powerful inexpensive cameras and related software to provide commodity eye-tracking. The method also leverages collected data on user interactions and uses machine learning techniques to customize its response to individual usage patterns. A system is specified for implementing the described method on a streaming architecture.