Multi-sensor Handle Controller Hybrid Tracking via Extended Kalman Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing handle controller tracking methods in VR/AR/MR are vulnerable to environmental interference, leading to issues like drifting, jittering, and jamming, which affect user experience.
Innovation Solution
A multi-sensor hybrid tracking method using optical, electromagnetic, and inertial navigation data, combined with an extended Kalman filtering fusion, to enhance tracking stability and reduce environmental interference.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a camera is used to track the handle controller through optical pattern mark points, then real-time 6DoF position and orientation information can be obtained, but the tracking performance is greatly influenced by ambient light complexity and mark point overlap
Solution Approach 1:
The patent combines multiple sensing modalities (optical camera tracking, electromagnetic sensor tracking, and IMU inertial navigation) into a unified tracking system. The camera captures optical pattern mark points for 6DoF information, while the electromagnetic sensor and IMU provide complementary data. By merging these different sensing approaches, the system compensates for the camera's vulnerability to ambient light interference, as the other sensors continue to function reliably under the same environmental conditions.
2Measurement precision
If an electromagnetic sensor is used to track the handle controller, then tracking data can be obtained, but the sensor is vulnerable to interference from complex electromagnetic signals in the environment
Solution Approach 1:
The patent integrates the electromagnetic sensor with other tracking modalities (camera-based optical tracking and IMU inertial navigation) to form a hybrid tracking system. When the electromagnetic sensor encounters interference from complex electromagnetic signals, the camera and IMU continue to provide reliable tracking data. The system fuses these multiple data sources to maintain accurate tracking performance despite environmental electromagnetic interference.
Solution Approach 2:
The system implements sensor fusion with feedback mechanisms where the camera and IMU provide corrective information when the electromagnetic sensor experiences interference. The camera captures optical pattern mark points to verify and correct position and orientation data, while the IMU provides inertial reference data. This feedback loop allows the system to detect and compensate for electromagnetic interference in real-time.
3Device complexity
If single-sensor tracking methods are used, then device complexity is reduced, but tracking stability deteriorates due to drifting, jittering, and jamming phenomena
Solution Approach 1:
The patent merges multiple sensing systems (camera, electromagnetic sensor, IMU) into a coordinated hybrid tracking system. Each sensor type has different strengths and weaknesses, and by combining them, the system achieves superior tracking stability compared to single-sensor approaches. The camera provides visual tracking, the electromagnetic sensor offers rapid response, and the IMU provides inertial reference, creating a stable and reliable tracking solution.
Solution Approach 2:
The patent applies the concept of composite materials to sensing systems by combining different sensor types into a composite tracking system. Just as composite materials combine different substances to achieve superior properties, this system combines optical, electromagnetic, and inertial sensors to achieve tracking stability that exceeds any individual sensor type. The fused data from these diverse sensors creates a robust tracking solution resistant to drifting, jittering, and jamming.
Data Source
Figure 1~2
Figure 3
AI summary
The present disclosure discloses a multi-sensor handle controller hybrid tracking method and device, wherein the method includes: acquiring tracking data of a handle controller, the tracking data including optical tracking data, electromagnetic tracking data, and inertial navigation data; constructing a state transition model and an observation model of an extended Kalman filtering iteration strategy according to the tracking data, and performing an extended Kalman filtering fusion on the optical tracking data, the electromagnetic tracking data, and the inertial navigation data; and determining position and orientation information of the handle controller in a space according to the extended Kalman filtering iteration strategy. According to the present disclosure, hybrid tracking on the handle controller is performed based on an optical sensor, an electromagnetic sensor, and an inertial navigation sensor, so that stability of tracking of the handle controller can be optimized on the premise of maintaining high-precision tracking quality of the handle controller.