Handheld Controller 6DOF Tracking with Passive Optical and Inertial Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems for mixed reality, such as HMD devices, face challenges in accurately tracking and disambiguating multiple wireless hand-held controllers with passive optical and inertial tracking, particularly in reducing processing overhead and achieving high resolution and accuracy in 6DOF tracking within virtual and augmented reality environments.
Innovation Solution
A wireless hand-held controller with a slim form-factor incorporating passive optical and inertial tracking, featuring two optically reflective markers and an IMU, which combines optical position tracking with inertial orientation tracking to provide accurate 6DOF data, reducing processing overhead and improving detection and segmentation of controller orientation and location.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If passive optical tracking is used for multiple controllers, then the system can achieve high resolution and accuracy in tracking, but the processing overhead increases significantly making it difficult to track and disambiguate multiple controllers
Solution Approach 1:
The patent segments the tracking problem by dividing controllers into trackable and non-trackable sets based on marker visibility. The optical tracking system only processes markers that are currently visible in the camera field of view, while inertial tracking handles controllers with occluded markers. This segmentation reduces processing overhead by avoiding attempts to optically track invisible markers while maintaining continuous 6DOF tracking accuracy.
Solution Approach 2:
The patent introduces inertial measurement units (IMUs) as intermediaries to bridge the gap between optical tracking limitations and continuous 6DOF tracking requirements. The IMU data serves as a mediator that provides orientation information when optical markers are not visible, allowing the system to maintain accurate tracking without the processing overhead of continuous optical marker detection for all controllers.
2Reliability
If optical markers are placed on controllers for tracking, then the system can detect controller position and orientation, but the markers may be occluded or outside the camera field of view making disambiguation difficult
Solution Approach 1:
The patent merges optical tracking and inertial tracking systems into a unified 6DOF tracking solution. The optical system provides precise position and orientation when markers are visible, while the inertial system continuously provides orientation data. By combining these two systems and fusing their data, the patent achieves reliable marker detection when visible while maintaining continuous tracking reliability through inertial compensation when markers are occluded or out of view.
Solution Approach 2:
The patent performs preliminary classification of controllers into trackable and non-trackable categories based on current marker visibility. This preliminary action allows the system to optimize processing by applying optical tracking only to visible markers while using inertial tracking for occluded ones, thereby improving both marker detection reliability and reducing the difficulty of detecting and measuring controller states in all conditions.
3Shape
If the controller form-factor is made slim, then the device is more comfortable and aesthetically pleasing, but the placement of optical markers and inertial sensors becomes more constrained
Solution Approach 1:
The patent implements the nested doll principle by placing the inertial measurement unit (IMU) inside the slim controller body, nested within the available internal space. The optical markers are positioned on the exterior surface of the slim form-factor controller. This nested arrangement allows the controller to maintain its slim, comfortable shape while still accommodating both the inertial sensors and optical markers necessary for 6DOF tracking, resolving the conflict between form-factor and ease of manufacture.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This solution enables robust and accurate tracking of multiple controllers with reduced processing overhead, achieving high resolution and accuracy in 6DOF, enhancing interaction in virtual and augmented reality environments by combining optical and inertial data for precise controller positioning and orientation.
Implementation Method 1
two passive optically reflective markers, one marker being position at or adjacent each end of the controller
Implementation Method 2
each controller also including an onboard inertial measurement unit for providing inertial data corresponding to its orientation
Data Source
AI summary
Methods for disambiguation and tracking of two or more wireless hand-held controllers with passive optical and inertial tracking within a system having a head mounted virtual or augmented reality display device having a forward facing optical sensor having a field of view, and wherein the display device interfaces with wireless hand-held inertial controllers for providing user input to the display device, with each controller two passive optically reflective markers, one marker being position at or adjacent each end of the controller and being separated by a known distance, and each controller also including an onboard inertial measurement unit for providing inertial data corresponding to its orientation.