EEG-Based VR Video Streaming Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current VR video streaming technologies face challenges in reducing motion-to-photon latency and optimizing bandwidth usage due to the need to stream large amounts of high-quality video data, especially when the user's field of view changes, leading to delays and reduced quality in immersive experiences.
Innovation Solution
The use of EEG data to predict head movements allows for an early adaptation of the streaming process by determining a second configured setup, ensuring that target image data is available just in time for the future view, reducing delays and optimizing bandwidth by focusing on relevant parts of the scene that will be viewed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If the entire panoramic VR video is streamed in high quality and 3D, then the visual quality and immersion are improved, but the bandwidth requirement increases to tens or hundreds of Mbps
Solution Approach 1:
The patent segments the VR video stream into multiple sub-streams corresponding to different regions of interest (e.g., foveal region, peripheral regions). Each sub-stream is encoded and transmitted separately, allowing selective allocation of bandwidth to different spatial regions based on their importance and the user's current viewing direction.
Solution Approach 2:
The patent applies local quality enhancement by transmitting high-quality video data only for the current field of view and regions predicted to be viewed next, while using lower quality or compressed data for peripheral or less important regions. This maintains visual quality where needed while reducing overall bandwidth consumption.
2Quantity of substance
If spatial segmentation is used to stream only visible parts of the VR video, then the bandwidth usage is reduced, but the motion-to-photon latency increases due to switching delays
Solution Approach 1:
The patent performs preliminary actions by predicting future head movements using EEG data and pre-loading or pre-processing the corresponding video regions before the user actually looks at them. This anticipatory approach ensures that when the user turns their head, the required video data is already prepared and available, reducing switching latency.
Solution Approach 2:
The patent introduces an intermediary prediction mechanism that acts as a buffer between user intent (measured by EEG) and the actual video stream switching. The prediction system serves as an intermediary that prepares transition data in advance, smoothing the transition between different video regions and reducing perceptible latency.
3Loss of time
If guard bands are used to provide video content outside the current view, then the switching latency is reduced, but the bandwidth available for visible video content decreases
Solution Approach 1:
The patent applies partial action by transmitting guard band data only in the directions and extents predicted to be necessary based on EEG-based movement prediction. Rather than providing full 360-degree guard bands, the system selectively provides partial guard bands only where and when needed, optimizing the trade-off between latency reduction and bandwidth consumption.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach significantly reduces motion-to-photon latency and improves the perceived quality of VR experiences by ensuring that the necessary image data is available before the head movement occurs, allowing for a more efficient use of bandwidth and processing power.
Implementation Method 1
a brain signal detector (115) for measuring brain signals
Implementation Method 2
predicting a head movement based on the EEG data
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Methods and devices are provided for use in a streaming process (130) of a Virtual Reality [VR] video to a VR rendering device (120). The streaming process has a configurable setup, each configured setup providing respective different image data of a scene. The VR rendering device is arranged to render a current view of the scene based on a current head position. The streaming process is executed according to a first configured setup providing first image data needed to render the current view. EEG data is determined by measuring brain signals (115) and a head movement is predicted based on the EEG data. Target image data is determined as needed for rendering a future view based on the predicted head movement. A new configured setup for providing the target image data is determined. By adapting the streaming process based on the prediction the target image data can be provided in time and efficiently using available bandwidth.