Shared Map Controller Tracking in AR Headsets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented and virtual reality systems face challenges in accurately tracking hand-held controllers, especially when they are not visible to the cameras, leading to inaccurate positioning and increased computational requirements.
Innovation Solution
A mapping and localization system that uses a head-mounted display (HMD) and controllers equipped with cameras to capture images of the environment, generating a shared map and leveraging a SLAM process to accurately determine the position of controllers, even when they are not visible, by processing images and feature points to reduce computational load on the controllers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a SLAM process is used to track controllers in augmented and virtual reality systems, then the position of controllers can be determined, but the tracking accuracy deteriorates when controllers are not visible to the cameras
Solution Approach 1:
The patent introduces an intermediary approach by using the HMD's camera system and shared environment map as a mediator to track controllers. Instead of relying solely on the controller's own camera when invisible, the system uses the HMD's captured images and generated map to indirectly determine controller position, improving tracking reliability during invisible periods
Solution Approach 2:
The system creates a copy of the environment through the shared map generated by the HMD's SLAM process. This digital copy allows the system to track controller positions by matching features against the pre-captured environment, enabling accurate tracking even when the controller camera cannot directly observe the surroundings
2Measurement precision
If controllers perform full image processing and SLAM independently, then tracking can be performed, but the computational load and processing requirements increase significantly
Solution Approach 1:
The patent merges the SLAM processing functionality between the HMD and controllers. The HMD performs the computationally intensive tasks of capturing images, generating the shared environment map, and running the SLAM process. Controllers then leverage this pre-processed shared map for lighter-weight position determination, significantly reducing their computational load while maintaining tracking precision
Solution Approach 2:
The shared environment map serves multiple functions: it provides the foundation for SLAM processing, enables controller position determination, and facilitates augmented reality rendering. By creating this universal data structure, the system eliminates redundant processing and allows different devices to use the same processed information for their respective functions
3Adaptability or versatility
If each device generates its own environment map independently, then device autonomy is maintained, but positioning consistency across devices deteriorates
Solution Approach 1:
The patent merges the environment mapping function into a single shared map generated by the HMD that is accessible to all controllers. This unified map ensures that all devices reference the same environmental features and coordinate system, guaranteeing positioning consistency across the system while the HMD maintains its autonomous mapping capability
Data Source
AI summary
A mapping and localisation apparatus includes: a communication processor configured to receive data relating to one or more first images of an environment captured by one or more first cameras of a first mobile electronic device; and receive data relating to one or more second images of the environment captured by one or more second cameras of one or more second mobile electronic devices; a mapping processor configured to process the data relating to the one or more first images to generate a three dimensional map of the environment; and a position processor configured to determine a position of the or each second mobile electronic device at least in part based on the generated map and the received data relating to the one or more second images.