Line-Locked AR Display Reducing Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional augmented reality (AR) display systems experience significant latency between image capture and display due to the need to buffer and process entire images before displaying, which is particularly noticeable in motion-based applications like head-mounted displays, leading to a perceptible lag and detracting from the user experience.
Innovation Solution
Implementing a 'line-locked' display system where each pixel row is processed and displayed as it is output by the imaging sensor, using a small buffer to temporarily store pixel rows and perform modifications, thereby reducing the need to buffer the entire image and minimizing latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If the entire image is buffered and processed before display in conventional AR systems, then complete image processing can be performed, but significant latency is introduced between image capture and display
Solution Approach 1:
The patent segments the image processing into line-by-line operations rather than processing the entire image as a single unit. Each line of pixels is processed independently as it is captured by the imaging sensor, allowing continuous processing without waiting for complete frame buffering. This segmentation eliminates the latency inherent in conventional full-frame buffering approaches.
Solution Approach 2:
The patent performs preliminary processing actions on pixel lines immediately upon their capture by the imaging sensor, before the complete image frame is available. By processing each line as it arrives and preparing display data in advance, the system minimizes the time delay between capture and display without requiring complex full-frame buffering infrastructure.
2Loss of time
If a small buffer is used to store only pixel rows instead of entire images, then latency is reduced, but processing continuity must be maintained
Solution Approach 1:
The patent maintains continuous processing action by immediately processing each pixel line as it is captured from the imaging sensor, without interruption or waiting for complete frames. The display controller continuously receives and processes pixel line data in real-time, ensuring uninterrupted processing flow that simultaneously reduces latency and maintains reliability through consistent operational state.
Solution Approach 2:
The patent introduces a display controller as an intermediary component that receives pixel line data directly from the imaging sensor and performs real-time processing and conversion to display format. This intermediary enables continuous processing with minimal buffering by mediating between the sensor output and display input, maintaining processing reliability while minimizing the buffer size required.
3Loss of time
If pixel lines are processed and displayed as they are output by the imaging sensor, then latency is minimized, but synchronization between capture and display must be precise
Solution Approach 1:
The patent implements a feedback mechanism where the display controller continuously monitors the pixel line data stream from the imaging sensor and dynamically adjusts processing timing to maintain synchronization. This feedback loop ensures that each processed line corresponds accurately to its captured moment, achieving precise synchronization while maintaining minimal latency through real-time adaptive timing control.
Data Source
AI summary
A method includes sequentially outputting from an imaging sensor each pixel row of a set of pixel rows of an image captured by the imaging sensor. The method further includes displaying, at a display device, a pixel row representative of a first pixel row of the captured image prior to a second pixel row of the captured image being output by the imaging sensor. An apparatus includes an imaging sensor having a first lens that imparts a first type of spatial distortion, a display device coupled to the imaging sensor, the display to display imagery captured by the imaging sensor with the first spatial distortion, and an eyepiece lens aligned with the display, the eyepiece lens imparting a second type of spatial distortion that compensates for the first type of spatial distortion.