Gaze-Based Video Pre-Filtering for Bandwidth-Limited XR Streaming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Managing the flow of video data over constrained network pathways while maintaining high user satisfaction is challenging due to limited bandwidth and varying video fidelity needs in applications like mixed and virtual reality.
Innovation Solution
Implementing gaze direction-based multi-layer low-pass filtering to reduce data transmission by using higher resolution settings for central gaze areas and lower settings for peripheral regions, combined with blending techniques to minimize visual artifacts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If video data is transmitted at high resolution to maintain video quality, then user satisfaction is improved, but network bandwidth consumption increases
Solution Approach 1:
The patent applies different filtering resolutions to different regions of the video frame based on the user's gaze direction. The central foveal region (where the user is looking) is processed at high resolution to maintain perceived quality, while peripheral regions are processed at lower resolution. This local differentiation allows the system to reduce overall bandwidth consumption while maintaining video quality in the visually critical areas.
2Quantity of substance
If gaze-based selective filtering is implemented, then bandwidth efficiency is improved, but system complexity increases
Solution Approach 1:
The patent segments the video frame into multiple regions (central foveal region and peripheral regions) and applies different filtering operations to each segment. This segmentation is based on the user's gaze direction, which divides the frame into a high-priority central area and lower-priority peripheral areas. The segmentation approach enables bandwidth efficiency through selective filtering while keeping the system relatively simple by using straightforward regional division and differential processing.
Data Source
AI summary
A multi-layer low-pass filter is used to filter a first frame of video data representing at least a portion of an environment of an individual. A first layer of the filter has a first filtering resolution setting for a first subset of the first frame, while a second layer of the filter has a second filtering resolution setting for a second subset. The first subset includes a data element positioned along a direction of a gaze of the individual, and the second subset of the frame surrounds the first subset. A result of the filtering is compressed and transmitted via a network to a video processing engine configured to generate a modified visual representation of the environment.