Peripheral Tracking via Interleaved Camera Images
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing motion tracking systems for peripherals in virtual and augmented reality environments face challenges with accuracy due to sensor drift and occlusions, particularly when relying on hardware motion detectors like accelerometers, which can lead to significant position errors.
Innovation Solution
A camera-based system mounted on a head-mountable display (HMD) captures interleaved images with different parameters to track both the HMD and peripherals, using image analysis to derive accurate motion and position data, potentially supplemented by SLAM techniques and hardware sensors, to improve tracking accuracy and reduce complexity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If hardware motion detectors (accelerometers) are used to track peripheral position, then the tracking system can provide motion detection capability, but the measurement precision deteriorates due to sensor drift accumulating errors quickly
Solution Approach 1:
The patent combines camera-based tracking with hardware sensor tracking to create a hybrid system. The camera provides absolute position reference while hardware sensors provide continuous motion data. By merging these two tracking methods, the system achieves both motion detection capability and high position accuracy, eliminating the drift problem of standalone sensor systems.
Solution Approach 2:
The camera acts as an intermediary reference system that corrects the drift accumulated by hardware motion detectors. Instead of relying solely on sensors that accumulate errors, the camera periodically provides absolute position information that resets and corrects the drift, maintaining long-term tracking accuracy.
2Measurement precision
If a camera-based tracking system is implemented, then the absolute position measurement precision is improved, but the device complexity increases due to additional cameras and processing requirements
Solution Approach 1:
The HMD camera serves multiple functions: it tracks the HMD's own position and orientation, tracks peripheral positions, and provides reference data for correcting sensor drift. This multi-functionality eliminates the need for separate tracking cameras, reducing overall system complexity while maintaining high measurement precision.
Solution Approach 2:
The HMD uses its own built-in camera to perform tracking functions that would otherwise require external tracking equipment. The camera mounted on the HMD captures images of the environment and peripherals, enabling the system to self-track without requiring additional external cameras or infrastructure.
3Measurement precision
If images are captured at high frame rates for accurate tracking, then the measurement precision is improved, but the productivity decreases due to increased processing time and latency
Solution Approach 1:
The patent divides image processing into separate stages: initial image capture, feature detection, position calculation, and drift correction. By segmenting the processing pipeline, the system can efficiently handle high frame rates while maintaining accuracy. Critical processing steps are prioritized, and less critical steps are optimized or deferred.
Solution Approach 2:
The system performs preliminary processing of images by detecting key features and calculating positions before full drift correction is applied. This preliminary action provides immediate tracking feedback while the more computationally intensive correction processes run in parallel or are applied in subsequent frames, maintaining high throughput.
Data Source
Figure 1~2b
Figure 3~4b
Figure 5~6
AI summary
A system for tracking one or more peripherals operable by a user in a real environment, the system comprising a camera, mountable upon the user, operable to capture images of the real environment, a feature identification unit operable to identify features in images belonging to a first set of images captured by the camera, a peripheral identification unit operable to identify the image position of one or more peripherals in one or more images belonging to a second set of images captured by the camera, and a position determining unit operable to generate first position information for the camera in the real environment in dependence upon one or more of the identified features, and to generate respective second position information for the one or more peripherals relative to the camera in dependence upon the identified image position of the one or more peripherals, wherein the first set of images and the second set of images captured by the camera are interleaved such that groups of one or more images from each set are captured in turn, and wherein the position determining unit is operable to generate third position information identifying the position of the one or more peripherals in the real environment by combining the second position information corresponding to a given group of images in the second set of images with a function of the first position information corresponding to a group of images in the first set of images preceding the given group of images and the first position information corresponding to a group of images in the first set of images following the given group of images .