AR Headset Noise Reduction via Region-Aware Frame Averaging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Augmented reality head-mounted displays (HMDs) face motion blur issues due to small pixel receptor areas and slow shutter speeds, which detract from the user experience, especially in low-light conditions where few photons reach the camera sensors, making fast shutter speeds impractical.
Innovation Solution
A head-mounted display system that uses eye tracking to identify and average image frames only in a specific 50×50 pixel region of interest, rotating and translating previous frames to match the current frame, thereby enhancing image quality and reducing noise and motion blur.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If the shutter speed is increased to reduce motion blur, then image quality improves, but the camera cannot capture enough photons in low-light conditions
Solution Approach 1:
The patent combines multiple frames captured at different time points to create a single enhanced image. By merging the information from multiple frames, the system accumulates sufficient photon data while maintaining a fast shutter speed to prevent motion blur in each individual frame.
Solution Approach 2:
The system performs preliminary actions by capturing multiple frames in advance before generating the final enhanced image. These pre-captured frames are stored and later processed through alignment and merging operations to produce the final result with reduced motion blur and improved signal-to-noise ratio.
2Illumination intensity
If the shutter speed is decreased to capture more photons, then light capture improves, but motion blur increases and detracts from reality
Solution Approach 1:
Multiple frames captured with increased exposure time are merged together to accumulate sufficient light signal. The merging process includes alignment transformations to compensate for motion, ensuring that the combined image maintains sharpness while benefiting from increased photon capture in each frame.
Solution Approach 2:
The system creates multiple copies of the scene at different time points and processes these copies through alignment and merging. By working with copies rather than the original single frame, the system can experiment with different exposure times and processing methods to optimize both light capture and motion blur reduction.
3Illumination intensity
If the pixel receptor area is increased to improve light capture, then photons received improve, but device size and complexity increase
Solution Approach 1:
Instead of using a single large pixel receptor, the system segments the imaging task across multiple smaller pixels that capture frames over time. By dividing the temporal dimension into multiple discrete frame captures, the system achieves equivalent light gathering capability without requiring larger individual pixels or a larger sensor.
Solution Approach 2:
The patent transitions from spatial dimension (larger pixels) to temporal dimension (multiple frames over time) to achieve the same light capture goal. By utilizing the time dimension through multi-frame capture and merging, the system avoids increasing the physical size of the camera sensor while still improving photon reception.
4Reliability
If multiple frames are averaged to reduce noise, then signal-to-noise ratio improves, but processing complexity and time increase
Solution Approach 1:
The system applies frame merging and averaging operations selectively to specific regions of interest rather than processing the entire image uniformly. By identifying and prioritizing certain areas for enhanced processing, the system improves signal-to-noise ratio in critical regions while reducing overall computational complexity compared to full-frame processing.
Solution Approach 2:
The image processing is segmented into distinct stages: frame capture, alignment transformation, region identification, selective merging, and final composite generation. This segmentation allows the system to apply computationally intensive averaging operations only where necessary, rather than uniformly across the entire image, thereby managing processing complexity more effectively.
Data Source
AI summary
Video noise reduction for a video augmented reality system is provided. A head mounted display includes a display unit; a camera for generating frames of display data. A frame store is provided for storing previous frames of displayed information that was sent to the display unit; and a motion processor is provided in communication with the camera, display unit, and the frame store. The motion processor is operable to: identify an area of interest in a current frame of display data; match the area of interest to similar areas in previous frames stored in the frame store; rotate and translate the matched areas of interest from the one or more previous frames stored in the frame store to match the area of interest in the current frame; and average the prior matched areas of interest with the current area of interest to generate a displayed area of interest.