Untethered Input Detection for Occlusion-Resistant Object Engagement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing electronic devices with integrated input systems, such as extremity tracking or eye tracking, face issues like tracking inaccuracies and limited user control when manipulating computer-generated objects, especially due to occlusions and limited mobility, which affect the reliability and efficiency of interaction.
Innovation Solution
The implementation of an untethered input detection system that combines finger manipulation data from a finger-wearable device with untethered inputs, allowing for six degrees of freedom tracking and independent object engagement based on finger manipulation data, regardless of the apparent distance between the device and the object, enhancing user control and accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If integrated input systems (extremity tracking or eye tracking) are used to manipulate computer-generated objects, then user interaction is enabled, but tracking inaccuracies and limited user control occur due to occlusions and limited mobility
Solution Approach 1:
The system divides the input mechanism into two independent segments: a finger-wearable device that always stays with the user (providing stable finger position data) and a separate physical proxy object that the user holds for interaction (providing spatial information). This segmentation allows each component to compensate for the other's limitations, maintaining tracking accuracy even when occlusions occur.
Solution Approach 2:
The physical proxy object acts as an intermediary between the user's finger movements and the computer-generated objects. Instead of directly tracking the finger through occlusions, the system uses the proxy object's position and orientation (captured by cameras) as a mediator to infer the user's intended interaction target, thereby maintaining reliable tracking accuracy.
2Productivity
If traditional integrated input systems are used, then interaction with computer-generated objects is possible, but the system is highly sensitive to occlusions by physical objects
Solution Approach 1:
The system converts the harmful effect of occlusions into a beneficial feature by using the physical proxy object as an intentional part of the interaction mechanism. The proxy object's presence (which would normally cause occlusion) is now exploited to provide spatial context and identify interaction targets, transforming the occlusion problem into an advantage for accurate object engagement.
Solution Approach 2:
The system creates a visual copy or representation of the physical proxy object in the augmented reality display. This copy allows the system to track the proxy object's position and orientation independently of actual occlusions, enabling continuous interaction even when the physical object blocks the view of the finger-wearable device.
3Ease of operation
If extremity tracking is used for object manipulation, then user interaction is achieved, but limited mobility and occlusions reduce tracking reliability
Solution Approach 1:
The system dynamically adapts the tracking method based on the interaction context. When the finger-wearable device is visible, it uses direct finger tracking for fine manipulation control. When occlusions occur, it dynamically switches to using the physical proxy object's position and orientation to maintain tracking reliability, ensuring continuous reliable operation throughout the interaction.
Data Source
AI summary
A method includes displaying a plurality of computer-generated objects, and obtaining finger manipulation data from a finger-wearable device via a communication interface. In some implementations, the method includes receiving an untethered input vector that includes a plurality of untethered input indicator values. Each of the plurality of untethered input indicator values is associated with one of a plurality of untethered input modalities. In some implementations, the method includes obtaining proxy object manipulation data from a physical proxy object via the communication interface. The proxy object manipulation data corresponds to sensor data associated with one or more sensors integrated in the physical proxy object. The method includes registering an engagement event with respect to a first one of the plurality of computer-generated objects based on a combination of the finger manipulation data, the untethered input vector, and the proxy object manipulation data.