Handheld Device Tracking in VR via Sensor Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current augmented and virtual reality systems face challenges in accurately tracking the movement of handheld electronic devices within immersive environments, particularly due to reflective surfaces and varying lighting conditions, which affect the precision of depth camera imaging.
Innovation Solution
A system that combines data from a depth camera on a head-mounted display with inertial measurement unit data from the handheld device, allowing for accurate 6DOF tracking of the device's position and orientation, and translates this data into corresponding actions within the virtual environment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a depth camera is used to track the handheld device, then the system can capture position data, but reflective surfaces and varying lighting conditions reduce measurement precision
Solution Approach 1:
The patent combines data from multiple sources: depth camera position data, accelerometer data, and orientation data from the handheld device. This multi-sensor fusion approach compensates for the depth camera's sensitivity to lighting conditions by relying on the inertial sensors within the handheld device, which are不受 lighting affects.
Solution Approach 2:
The system uses the handheld device itself as an intermediary tracking target rather than relying solely on external markers or features in the environment. The device's own sensors (accelerometer and orientation sensors) serve as intermediaries to provide reliable tracking data independent of environmental lighting conditions.
2Measurement precision
If multiple sensors are combined for tracking, then measurement precision improves, but device complexity increases
Solution Approach 1:
The handheld device serves multiple functions: it acts as both the user interface and the tracking target. The same device contains both the sensors for navigation (accelerometer, orientation sensors) and the target features for camera tracking, eliminating the need for separate specialized tracking equipment.
Solution Approach 2:
The handheld device tracks itself by utilizing its own embedded sensors (accelerometer and orientation sensors) to provide position and orientation data. This self-service capability reduces system complexity by eliminating the need for external tracking infrastructure.
3Reliability
If specialized tracking equipment is used, then tracking reliability improves, but the system requires additional hardware
Solution Approach 1:
The handheld device is designed to be self-sufficient for tracking purposes by incorporating its own sensors (accelerometer and orientation sensors). This eliminates the need for specialized external tracking equipment while maintaining reliable tracking through the device's intrinsic capabilities.
Solution Approach 2:
The system uses the handheld device for multiple purposes: as the user interface, as the navigation sensor platform, and as the tracking target. This multi-functionality eliminates the need for separate specialized tracking hardware.
Data Source
Figure 1
Figure 2A~2B
Figure 3
AI summary
A system for tracking a first electronic device, such as a handheld electronic device, in a virtual reality environment generated by a second electronic device, such as a head mounted display may include the fusion of data collected by sensors of the electronic device with data collected by sensors of the head mounted display, together with data collected by a front facing camera of the electronic device related to the front face of the head mounted display.