Stereo Camera Calibration Using Handheld Light Sources
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems for tracking the pose of handheld objects in augmented and virtual reality environments rely on external light sources or cameras, which require setup and can be cumbersome, and do not efficiently combine optical and inertial data for accurate pose determination.
Innovation Solution
A head-mounted device with a stereo camera arrangement and inertial measurement units that detect light sources on handheld objects, using image data and IMU information to fuse optical and inertial data for precise pose tracking without external setup, employing methods like Kalman filters and light pulse synchronization for accurate calibration and tracking.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If external light sources or cameras are used for tracking handheld objects, then pose tracking can be achieved, but the system requires external setup and becomes cumbersome
Solution Approach 1:
The patent combines the tracking functionality into the head-mounted device itself by integrating a camera and light source into a single self-contained unit. The camera is mounted on the HMD and works with light sources that can be either integrated into the handheld object or provided by the HMD, eliminating the need for separate external tracking systems and reducing overall system complexity.
Solution Approach 2:
The HMD serves multiple functions: it acts as both the tracking device (with integrated camera) and the light source for illuminating handheld objects. This multi-functional design eliminates the need for dedicated external tracking equipment, allowing the same device to perform both imaging and illumination tasks required for pose tracking.
2Measurement precision
If multiple light sources are used on handheld objects for better tracking, then pose determination accuracy improves, but power consumption increases
Solution Approach 1:
The light sources on handheld objects are controlled to flash or pulse at specific intervals rather than remaining continuously illuminated. This periodic activation provides sufficient light for the camera to capture images and determine pose accurately, while significantly reducing overall power consumption compared to continuous operation.
Solution Approach 2:
The system uses the handheld object's own light sources (when available) to illuminate itself for tracking purposes. The handheld object serves its own illumination needs without requiring external lighting equipment, and the controller manages the light activation based on tracking requirements to optimize power usage.
3Reliability
If light sources are continuously illuminated for optimal tracking, then tracking accuracy is maintained, but battery life is reduced
Solution Approach 1:
Light sources are activated in periodic pulses synchronized with the camera's imaging cycles rather than remaining continuously on. This ensures that light is available whenever the camera needs to capture images for tracking, maintaining reliability, while the intermittent operation pattern extends battery life by eliminating unnecessary continuous illumination.
Solution Approach 2:
The light source activation is dynamically controlled based on the actual tracking needs and camera operation. The controller adjusts when lights are turned on and off according to the imaging schedule and environmental conditions, optimizing the balance between maintaining sufficient illumination for accurate tracking and conserving battery power throughout the device's operational duration.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables immersive and accurate tracking of handheld objects within virtual and augmented reality environments by eliminating the need for external setup, improving pose determination accuracy through the fusion of optical and inertial data, and dynamically adjusting light source brightness for optimal tracking and power management.
Implementation Method 1
receive image data of a field of view of the camera, detect a plurality of light sources of a handheld object in the image data
Implementation Method 2
based upon the plurality of light sources detected, determine a pose of the handheld object... based upon the pose of the handheld object determined for each camera in the stereo camera arrangement
Data Source
Figure 1
Figure 2
Figure 3
AI summary
One disclosed example provides a head-mounted device including a stereo camera arrangement, a logic device configured to execute instructions, and a storage device storing instructions executable by the logic device to, for each camera in the stereo camera arrangement, receive image data of a field of view of the camera, detect light sources of a handheld object in the image data, and based upon the light sources detected, determine a pose of the handheld object. The instructions are executable to, based upon the pose of the handheld object determined for each camera in the stereo camera arrangement, calibrate the stereo camera arrangement.