Mixed Reality Camera Segmentation for Low Light IR Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing mixed-reality systems face challenges in performing computer vision tasks and image capture tasks under low light conditions, as sensors fail to achieve acceptable performance for both tasks simultaneously, particularly in environments with low light levels below 10 lux.
Innovation Solution
The implementation of a mixed-reality system with a first set of cameras configured to detect visible spectrum light and a particular band of infrared (IR) light for computer vision tasks, and a second set of cameras with detachable IR filters to attenuate IR light for image capture tasks, enabling accurate head tracking and pass-through imaging in IR-illuminated low light environments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single set of cameras is used for both computer vision tasks and image capture tasks, then the system structure is simplified, but performance for both tasks under low light conditions deteriorates
Solution Approach 1:
The camera system is segmented into two distinct sets: a first set of cameras configured for computer vision tasks with high sensitivity to IR light, and a second set of cameras configured for image capture tasks with reduced IR sensitivity. This segmentation allows each set to be optimized for its specific function, resolving the contradiction between system simplicity and task performance.
Solution Approach 2:
Different spectral sensitivity characteristics are applied to different parts of the camera system. The first set of cameras has enhanced IR sensitivity for accurate pose detection, while the second set has reduced IR sensitivity for realistic image capture. This local differentiation in optical properties enables both tasks to perform well under low light conditions.
2Measurement precision
If cameras are configured for high IR sensitivity to improve computer vision performance in low light, then pose detection accuracy is improved, but image capture quality deteriorates due to IR contamination
Solution Approach 1:
The imaging system is divided into two separate camera sets with different spectral sensitivities. The first set captures IR-rich images for accurate pose detection, while the second set captures IR-reduced images for realistic visual representation. This segmentation resolves the conflict between measurement precision and image quality.
Solution Approach 2:
Each camera set is configured with specific spectral sensitivity characteristics appropriate for its function. The first set has high IR sensitivity for measurement tasks, while the second set has reduced IR sensitivity for visualization tasks. This local optimization of optical properties enables both high precision measurement and high quality imaging.
3Illumination intensity
If cameras are configured for high IR sensitivity to capture more light in low light environments, then image brightness is improved, but color accuracy and realism deteriorate
Solution Approach 1:
The system uses two separate camera sets: one optimized for brightness (high IR sensitivity) and one optimized for color accuracy (reduced IR sensitivity). This segmentation allows the first camera to provide sufficient brightness in low light while the second camera maintains color realism, resolving the contradiction between illumination intensity and color accuracy.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This solution allows for accurate pose detection and realistic pass-through images in low light conditions, enhancing mixed-reality experiences by providing world-locked holograms and parallax-corrected views, and is applicable in various lighting conditions, including IR-illuminated environments.
Implementation Method 1
a first set of one or more cameras configured to detect at least a visible spectrum light and at least a particular band of wavelengths of infrared (IR) light
Implementation Method 2
a second set of one or more cameras that includes one or more detachable IR filters configured to attenuate IR light, including at least a portion of the particular band of wavelengths of IR light
Data Source
AI summary
Systems and methods are provided for facilitating computer vision tasks (e.g., simultaneous location and mapping) and pass-through imaging include a head-mounted display (HMD) that includes a first set of one or more cameras configured for performing computer vision tasks and a second set of one or more cameras configured for capturing image data of an environment for projection to a user of the HMD. The first set of one or more cameras is configured to detect at least a visible spectrum light and at least a particular band of wavelengths of infrared (IR) light. The second set of one or more cameras includes one or more detachable IR filters configured to attenuate IR light, including at least a portion of the particular band of wavelengths of IR light.