Head-Mounted Device Active Optical Foveation Camera
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Head-mounted devices face challenges in efficiently capturing and processing high-resolution images of the user's environment, leading to increased processing burden and power consumption, especially when displaying augmented reality content.
Innovation Solution
Implementing a pass-through camera with adjustable components such as image sensors with varying pixel densities, distortion lenses, and mirrors that selectively capture high-resolution image data only in the user's direct field of view, while capturing lower resolution data for peripheral vision, using gaze-tracking systems to determine the focus of attention.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the pass-through camera captures high-resolution images of the entire environment, then the image quality for augmented reality display is improved, but the processing burden and power consumption increase
Solution Approach 1:
The patent applies local quality by capturing high-resolution images only in the user's direct field of view (foveated region) while capturing lower-resolution images in peripheral regions. This is achieved through adjustable image sensors with varying pixel densities or optical elements that direct light selectively, thereby reducing overall processing burden and power consumption while maintaining perceptual quality where the user actually looks.
Solution Approach 2:
The patent segments the field of view into multiple regions based on user gaze data: a high-priority foveated region requiring high-resolution capture and lower-priority peripheral regions accepting lower resolution. This segmentation allows the system to allocate computational and optical resources efficiently, processing only the necessary high-resolution data for the relevant region rather than the entire scene.
2Measurement precision
If the pass-through camera captures high-resolution images of the entire environment, then the image quality for augmented reality display is improved, but the processing burden increases
Solution Approach 1:
The system implements local quality by directing high-resolution capture only to the foveated region of interest determined by gaze tracking, while using lower-resolution capture for peripheral regions. This reduces the total volume of high-resolution image data requiring processing, thereby lowering the processing burden on the device while maintaining quality where needed.
Solution Approach 2:
The patent segments the imaging task into different resolution requirements for different spatial regions. By processing only the foveated region at high resolution and peripheral regions at lower resolution, the system divides the computational workload strategically, reducing overall processing burden while preserving visual quality in the user's direct line of sight.
3Productivity
If adjustable components are used to selectively capture high-resolution data, then the processing efficiency is improved, but the device complexity increases
Solution Approach 1:
The patent employs dynamic adjustable components (such as movable image sensors with varying pixel densities, distortion lenses, or mirrors) that can change their configuration in real-time based on user gaze direction. This dynamic adaptability allows the system to optimize processing efficiency by capturing high-resolution data only when and where needed, while the added complexity of adjustable components is justified by the significant reduction in processing requirements.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach reduces processing burden and power consumption by optimizing image resolution based on the user's gaze, enhancing the performance and efficiency of head-mounted devices in displaying augmented reality content.
Implementation Method 1
a distortion lens that optically stretches a selected portion of the captured scene
Implementation Method 2
a deformable mirror that redirects light to direct a desired portion of the captured scene to the high pixel density region
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A pass-through camera in a head-mounted device may capture image data for displaying on a display in the head-mounted device. However, only low-resolution image data may be needed to display low-resolution images in the periphery of the user's field of view on the display. Therefore, the pass-through camera may only capture high-resolution images that correspond to the portion of the user's field-of-view that is being directly viewed and may capture lower resolution image data that corresponds to the real -world objects in the user's peripheral vision. To enable the camera module to selectively capture high-resolution images, the pass-through camera may include an image sensor with two or more pixel densities, a distortion lens, and/or one or more planar or curved mirrors. Any of the components in the camera module may be adjusted to change which portion of a scene is captured with high-resolution image data.