AR Handheld Pose Fusion via Discrepancy Verification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current augmented reality (AR), virtual reality (VR), and mixed reality (MR) systems face challenges in accurately determining whether a handheld device is being held by a user, which affects the precision of pose data fusion and user interaction, leading to potential errors in virtual object placement and interaction within dynamic environments.
Innovation Solution
The method involves capturing hand pose and handheld device pose data using sensors, determining pose discrepancies, and performing a fusion operation based on these discrepancies, including suppressing or augmenting data depending on distance and orientation thresholds to accurately determine if the device is being held, thereby enhancing the accuracy of pose data and user interaction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If hand pose data and handheld device pose data are fused without verification, then the system can operate continuously, but the accuracy of pose data deteriorates due to potential mismatches when the device is not being held
Solution Approach 1:
The system performs preliminary verification by calculating the distance between hand pose and device pose before fusing the data. This preliminary check determines whether the device is being held by the user, and only fuses data when the distance threshold is satisfied, preventing inaccurate data fusion while maintaining system reliability
Solution Approach 2:
The system continuously monitors the distance between hand pose and device pose, using this feedback to dynamically control the data fusion process. When the distance exceeds the threshold, the system stops fusing hand pose data, creating a feedback loop that maintains accuracy while adapting to changing user interactions
2Measurement precision
If the system continuously monitors hand pose and device pose to determine holding status, then the accuracy of interaction detection improves, but the computational load and processing time increase
Solution Approach 1:
The system performs partial verification by only checking the distance metric between hand and device poses rather than analyzing all pose parameters. This selective approach maintains high precision in detecting whether the device is being held while reducing computational overhead and processing time
3Measurement precision
If the system uses a strict distance threshold for fusion, then the accuracy of determining device-holding status improves, but the system becomes less adaptable to different user behaviors and device positions
Solution Approach 1:
The system dynamically adjusts the fusion operation based on real-time distance measurements. Rather than using a fixed strict threshold that rejects all borderline cases, the system continuously evaluates the current distance against the threshold and adapts the fusion operation accordingly, maintaining precision while accommodating natural variations in user behavior and device positioning
Data Source
AI summary
An augmented reality (AR) system includes a handheld device comprising handheld fiducials affixed to the handheld device. The AR system also includes a wearable device comprising a display operable to display virtual content and an imaging device mounted to the wearable device and having a field of view that at least partially includes the handheld fiducials and a hand of a user. The AR system also includes a computing apparatus configured to receive hand pose data associated with the hand based on an image captured by the imaging device and receive handheld device pose data associated with the handheld device based on the image captured by the imaging device. The computing apparatus is also configured to determine a pose discrepancy between the hand pose data and the handheld device pose data and perform an operation to fuse the hand pose data with the handheld device pose data.