VR Headset Gaze Filtering for Low-Bandwidth Panoramic Streaming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current broadcasting and streaming methods for panoramic events require high-capacity networks and intensive processing due to the high bandwidth needs of panoramic video signals, and lack flexibility in client-specific signal adaptation and network regulation.
Innovation Solution
A system utilizing a virtual-reality headset to determine gaze positions and generate content-filtered signals based on view regions, combined with a flexible streaming server for adaptive processing and network connectivity, allowing clients to select and adapt panoramic scenes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If panoramic video signals are transmitted directly from the video source to client devices, then the complete panoramic view is available to clients, but high bandwidth and high-speed processing capability are required
Solution Approach 1:
The panoramic video signal is segmented into multiple view regions or tiles that can be independently processed and transmitted. Instead of sending the complete high-bandwidth panoramic signal to all clients, the server divides the content into manageable segments that can be selectively transmitted based on client device characteristics and network conditions, thereby reducing overall bandwidth requirements while maintaining adaptability.
Solution Approach 2:
Different regions of the panoramic video signal are processed with different quality levels based on their importance and the specific client's requirements. The server identifies and prioritizes transmission of critical view regions while reducing quality or skipping less important regions for clients with limited bandwidth or processing capability, achieving client-specific adaptation without uniformly high bandwidth consumption.
2Adaptability or versatility
If a streaming server performs transcoding functions to adapt data according to client device characteristics, then signal adaptation to different devices is achieved, but processing intensity and server complexity increase
Solution Approach 1:
The server performs preliminary processing of the panoramic video signal by pre-segmenting it into view regions and pre-adapting different quality versions before actual client requests arrive. This preliminary action reduces the real-time processing burden when clients connect, as the server can simply select and transmit pre-processed segments rather than performing full transcoding for each client connection, thereby reducing server complexity while maintaining adaptability.
3Area of stationary object
If panoramic video signals are transmitted with high bandwidth capacity, then complete scene coverage is provided, but network resource consumption increases
Solution Approach 1:
The server extracts and transmits only the specific view regions that are relevant to each client's device characteristics and viewing requirements, rather than transmitting the complete panoramic scene. By taking out only the necessary portions of the panoramic content and transmitting those selectively, the system maintains adequate view region coverage for each client while significantly reducing network resource consumption compared to transmitting the full panoramic signal to all clients.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Gaze positions of an operator wearing a virtual-reality headset displaying a video stream define preferred view regions of the display. Starting with a reference gaze position, and for each subsequent distinctly different gaze position, the virtual-reality headset sends control data, including three spatial coordinates and a time coordinate expressed as a cyclical video-frame index, to a view adaptor receiving the video stream. The view adaptor stores contents of a number of most recent video frames of the video stream in a circular content-buffer and control data of a number of most recent gaze positions in a circular control-buffer. A content filter within the view adaptor determines a preferred view region surrounding a gaze position according to control data held in the circular control-buffer and extracts a partial content of a respective frame held in the circular content-buffer according to the preferred view region.