Passive Depth Extraction in Lightweight XR Headsets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Wearable cross reality (XR) systems face challenges with weight, power consumption, and accuracy in capturing stereoscopic depth information due to sensor shifts and high power requirements from active depth sensors, leading to unrealistic virtual object placement relative to physical objects.
Innovation Solution
A wearable XR system with a combination of a global shutter camera and a rolling shutter camera, along with a processor that performs compensation routines and size reduction, reduces power consumption by selectively using sensors and disabling or modulating their operation based on power and world model completeness criteria, while maintaining accurate stereoscopic depth information through calibration routines.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If active depth sensors are used to capture stereoscopic depth information, then measurement precision is improved, but use of energy increases
Solution Approach 1:
The patent extracts the depth sensing function from active depth sensors and implements it passively using existing camera systems. The processor creates a world model using depth information stereoscopically determined from images acquired by standard cameras, eliminating the need for power-intensive active depth sensors while maintaining depth measurement capability
Solution Approach 2:
The patent uses image copying and processing techniques to derive depth information. By capturing images from multiple cameras and processing them through compensation routines and world model creation, the system copies visual information into a three-dimensional representation without requiring active sensing hardware
2Measurement precision
If multiple sensors are used to capture accurate depth information, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The patent makes standard cameras multi-functional by using them for both 2D imaging and 3D depth sensing. The processor performs compensation routines and creates world models that enable depth perception from conventional image sensors, allowing these sensors to serve multiple purposes and reducing overall system complexity
Solution Approach 2:
The patent merges the depth sensing function with the existing camera system. By combining images from multiple cameras and processing them through a unified world model creation process, the system integrates depth perception into the standard imaging pipeline rather than requiring separate dedicated depth sensing hardware
3Measurement precision
If sensor operation is continuously active to maintain world model completeness, then measurement precision is improved, but use of energy increases
Solution Approach 1:
The patent implements periodic action by selectively enabling or disabling cameras based on power criteria and world model completeness assessments. The processor evaluates whether current depth information is sufficient and only activates additional sensing when necessary, creating a rhythm of active and inactive periods that conserves energy while maintaining adequate world understanding
Data Source
Figure 1
Figure 2
Figure 3A~3B
AI summary
A wearable display system including multiple cameras and a processor is disclosed. A greyscale camera and a color camera can be arranged to provide a central view field associated with both cameras and a peripheral view field associated with one of the two cameras. One or more of the two cameras may be a plenoptic camera. The wearable display system may acquire light field information using the at least one plenoptic camera and create a world model using the first light field information and first depth information stereoscopically determined from images acquired by the greyscale camera and the color camera. The wearable display system can track head pose using the at least one plenoptic camera and the world model. The wearable display system can track objects in the central view field and the peripheral view fields using the one or two plenoptic cameras, when the objects satisfy a depth criterion.