6DoF Controller Tracking via Depth Sensor Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current motion tracking technologies in virtual reality and augmented reality applications often limit user experience by not fully capturing six degrees of freedom, especially in environments with varying lighting conditions and close object distances, where traditional methods struggle to accurately track controllers with three degrees of freedom.
Innovation Solution
The implementation of an electronic device equipped with multiple imaging cameras and a depth sensor that uses modulated light patterns and multiview image analysis to determine the position and orientation of controllers in 3D space, combining depth data from the depth sensor with image and non-image sensor data to achieve six degrees of freedom tracking, even in challenging lighting conditions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional motion tracking methods are used, then the system is simpler to implement, but the tracking precision deteriorates in challenging lighting conditions and close object distances
Solution Approach 1:
The system segments the tracking function across multiple independent sensors (depth sensor, imaging cameras, inertial sensors) rather than relying on a single complex tracker. Each sensor type contributes specific data (depth, visual features, motion dynamics) that together enable precise 6DoF tracking while maintaining individual sensor simplicity.
Solution Approach 2:
The patent combines data from multiple sensor types (depth sensor data, image sensor data, inertial sensor data) through sensor fusion algorithms to achieve tracking precision that exceeds what any single sensor could provide alone, particularly in challenging lighting conditions and close distances.
2Adaptability or versatility
If six degrees of freedom tracking is implemented, then the user experience is improved, but the device complexity increases
Solution Approach 1:
The system uses universal sensor components (depth sensors, imaging cameras, inertial sensors) that can track multiple degrees of freedom simultaneously. These sensors are not dedicated to single functions but provide data that contributes to 6DoF tracking, reducing the need for specialized complex hardware.
Solution Approach 2:
The patent introduces an intermediary processing system that fuses data from multiple sensors and computes 6DoF pose information. This intermediary layer manages the complexity of coordinating multiple sensors while presenting a unified tracking output to the application layer.
3Measurement precision
If depth sensors and multiple imaging cameras are used, then the measurement precision is improved, but the use of energy increases
Solution Approach 1:
The system employs partial sensor activation or selective processing where not all sensors operate at full capacity simultaneously. The processing system adjusts the level of detail processed from each sensor based on current tracking requirements, reducing unnecessary energy consumption while maintaining precision when needed.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This solution enables precise and robust six degrees of freedom tracking of controllers, enhancing user experience in VR and AR applications by accurately capturing both positional and rotational movements, even in environments with limited lighting, without the need for additional sensors on the controller.
Implementation Method 1
uses modulated light patterns and multiview image analysis to determine the position and orientation of controllers in 3D space
Data Source
AI summary
A method for controller tracking with multiple degrees of freedom includes generating depth data at an electronic device based on a local environment proximate the electronic device. A set of positional data is generated for at least one spatial feature associated with a controller based on a pose of the electronic device, as determined using the depth data, relative to the at least one spatial feature associated with the controller. A set of rotational data is received that represents three degrees-of-freedom (3DoF) orientation of the controller within the local environment, and a six degrees-of-freedom (6DoF) position of the controller within the local environment is tracked based on the set of positional data and the set of rotational data.