Foveated Image Capture Using Direct Eye-Tracker ROI Sync
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing foveated image generation techniques in extended reality devices suffer from latency issues due to the processing and determination of the region of interest (ROI) at a processor, which can lead to photon-to-photon latency exceeding undesirable thresholds, causing discomfort for users.
Innovation Solution
The proposed method determines the ROI directly at an eye tracker and provides this information directly to a foveated-image sensor, bypassing memory and processor latency, allowing more synchronized and accurate foveated image capture.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If ROI determination is performed at a processor through memory, then processing capability is improved, but latency increases
Solution Approach 1:
The patent extracts the ROI determination function from the central processor and memory system, implementing it directly at the eye tracker device. This extraction eliminates the latency associated with data transfer to and processing through the processor-memory system, while the eye tracker maintains sufficient processing capability to determine ROI independently
Solution Approach 2:
The eye tracker acts as an intermediary device that directly receives sensor data and determines ROI without requiring processor intervention. This intermediary approach allows the system to maintain the processing capability of dedicated hardware while avoiding the latency bottleneck of the processor-memory architecture
2Device complexity
If ROI information is provided through processor and memory, then system integration is improved, but synchronization accuracy deteriorates
Solution Approach 1:
The patent extracts the ROI determination and provision functions from the central processor system and implements them directly at the eye tracker. This extraction maintains system integration through direct communication interfaces while dramatically improving synchronization accuracy by eliminating processor-mediated data paths
Solution Approach 2:
The system is segmented into independent functional modules: the eye tracker independently determines ROI and directly provides it to the foveated-image sensor. This segmentation allows each module to operate autonomously with high precision while maintaining overall system integration through defined communication interfaces
Data Source
AI summary
Systems and techniques are described herein for generating foveated image data. For instance, an apparatus for generating foveated image data is provided. The apparatus may include a memory; and a first processor communicatively connected directly to an image sensor and communicatively connected to the memory, the first processor configured to: determine an orientation of at least one eye of a user; store an indication of the orientation in the memory; and provide the indication of the orientation to the image sensor wherein the image sensor is configured to capture a foveated image based on the indication of the orientation, the foveated image comprising a first portion and a second portion, wherein the first portion is based on the indication of the orientation and has a first resolution, and wherein the second portion has a second resolution that is lower than the first resolution.