Video Pass-Through Computing System Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video pass-through computing systems suffer from high latency due to image signal processing being performed by a host computer, leading to an unpleasant AR experience with lag and artifacts.
Innovation Solution
The system performs image signal processing of the camera image pixel stream at a sub-frame level directly in hardware within the head-mounted display device, and uses feedback to temporally synchronize the real and virtual content, reducing latency and improving AR experience.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If image signal processing is performed by a host computer, then processing capability is sufficient, but latency becomes high causing lag and artifacts
Solution Approach 1:
The patent segments the image signal processing function from the host computer and relocates it to a dedicated processing circuit within the head-mounted display device. This separation allows the host computer to focus on high-level processing while the dedicated circuit handles real-time frame processing, thereby reducing latency without sacrificing processing capability.
Solution Approach 2:
The patent introduces a dedicated processing circuit as an intermediary between the camera and the display. This intermediary component processes image signals in real-time within the head-mounted display device, acting as a bridge that reduces the latency inherent in host computer processing while maintaining sufficient processing power.
2Device complexity
If camera exposure timing is not synchronized with virtual image pixel stream, then processing is simpler, but temporal synchronization is poor causing misalignment
Solution Approach 1:
The patent implements a feedback mechanism where the processing circuit determines whether camera image pixels are temporally synchronized with virtual image pixels. Based on this determination, the system adjusts the camera exposure timing to achieve proper synchronization. This feedback loop ensures accurate temporal alignment without requiring overly complex preprocessing.
Data Source
AI summary
A video pass-through computing system includes a head-mounted display device including a display, a camera configured to image a physical scene according to an exposure timing, and an augmented reality control circuit configured to receive a virtual image pixel stream and composite the camera image pixel stream with the virtual image pixel stream to generate a display image pixel stream output to the display, and if a corresponding pixel of the camera image pixel stream is not in temporal synchronization with a pixel of the virtual image pixel stream adjust the exposure timing of the camera.