Wearable Input Fusion for Accurate 3D Intent Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional input techniques in AR/VR/MR environments require high specificity and suffer from high error rates and user fatigue due to imprecise commands, making it challenging to accurately interact with virtual or real objects in 3D space.
Innovation Solution
A wearable device dynamically fuses multiple sensor inputs, such as head pose, eye gaze, hand gestures, and voice commands, to anticipate and predict user intent, dynamically adding or removing inputs based on convergence and divergence, reducing reliance on high-resolution sensors and enhancing interaction accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple sensor inputs are used to improve interaction accuracy, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The patent combines multiple sensor inputs (head pose, eye gaze, hand gestures, voice commands) into a unified input fusion system. The wearable device dynamically integrates data from various sensors to determine user intent, merging their outputs to achieve more accurate interaction detection than any single sensor could provide alone.
Solution Approach 2:
The input fusion system serves multiple functions: it processes different types of sensor data, dynamically adjusts input combinations based on context, and provides a universal interface for interacting with both virtual and physical objects. This multi-functional approach allows the same system to handle diverse interaction scenarios without requiring separate dedicated systems for each function.
2Measurement precision
If high-resolution sensors are used to ensure precise input detection, then measurement precision is improved, but device complexity and cost increase
Solution Approach 1:
The system combines multiple lower-resolution sensor inputs to achieve the effective precision of high-resolution sensors. By fusing data from head pose, eye gaze, hand gestures, and voice commands, the system compensates for individual sensor limitations and achieves accurate user intent detection without requiring any single sensor to have extremely high resolution.
Solution Approach 2:
The input fusion algorithm acts as an intermediary that processes and integrates signals from multiple sensors. This intermediary layer transforms individual sensor readings into a unified representation of user intent, effectively mediating between the imperfect individual sensor outputs and the desired precise interaction control.
3Measurement precision
If multiple input modes are continuously monitored to improve interaction accuracy, then measurement precision is improved, but use of energy increases
Solution Approach 1:
The system dynamically adjusts which sensor inputs are active and how they are weighted based on the current interaction context. Rather than continuously processing all sensor inputs at full resolution, the system adapts its processing intensity and input selection in real-time, reducing energy consumption when full precision is not required while maintaining accuracy when it is needed.
Solution Approach 2:
The input fusion system changes processing parameters such as the number of active sensors, the weighting of different input modes, and the resolution of processing based on contextual requirements. This allows the system to optimize between energy consumption and detection accuracy by adjusting parameters like sensor activation levels and processing depth according to the current interaction state.
Data Source
AI summary
Examples of wearable systems and methods can use multiple inputs (e.g., gesture, head pose, eye gaze, voice, totem, and/or environmental factors (e.g., location)) to determine a command that should be executed and objects in the three-dimensional (3D) environment that should be operated on. The wearable system can detect when different inputs converge together, such as when a user seeks to select a virtual object using multiple inputs such as eye gaze, head pose, hand gesture, and totem input. Upon detecting an input convergence, the wearable system can perform a transmodal filtering scheme that leverages the converged inputs to assist in properly interpreting what command the user is providing or what object the user is targeting.