Avatar-Based Gesture Interface for Depth Camera Ambiguity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
When using a depth-sensing camera for gesture-based inputs, there is spatial ambiguity in mapping user motions to cursor movements on a user interface due to factors like arm length, orientation, and distance from the camera, making calibration time-consuming and disruptive in dynamic environments.
Innovation Solution
Presenting an avatar on the display that represents the user and tracks their motions, allowing users to adjust their interactions based on the avatar's responses, thereby avoiding the need for calibration and clarifying intended user interface interactions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If depth-sensing camera is used for gesture-based inputs, then natural user interface interaction is enabled, but spatial ambiguity in mapping user motions to cursor movements occurs
Solution Approach 1:
The system displays an avatar that provides visual feedback to the user about how their motions are being interpreted and mapped to the user interface. The avatar mirrors the user's gestures and shows the corresponding UI control being activated, creating a closed-loop feedback system that helps users understand and adjust their motion-to-cursor mapping in real-time
Solution Approach 2:
The avatar serves as an intermediary between the user's physical motions and the digital user interface controls. It translates and visualizes the complex spatial mapping relationship, making the abstract connection between gesture and cursor position tangible and understandable for the user
2Measurement precision
If calibration is performed to resolve spatial ambiguity, then mapping precision is improved, but user interaction becomes time-consuming and disruptive
Solution Approach 1:
The system performs self-calibration by automatically adjusting the motion-to-cursor mapping based on feedback from the avatar and user responses, eliminating the need for manual calibration procedures. The avatar continuously adapts to the user's preferred interaction style through ongoing interaction rather than requiring a separate calibration phase
3Loss of information
If avatar provides detailed motion tracking feedback, then user interface interaction clarity is improved, but system complexity increases
Solution Approach 1:
The system extracts and displays only the most relevant aspects of user motion through the avatar, rather than showing all raw sensor data. It isolates and visualizes the key gesture components that map to UI controls, filtering out unnecessary complexity while preserving essential interaction information
Data Source
AI summary
Embodiments are disclosed that relate to interacting with a user interface via feedback provided by an avatar. One embodiment provides a method comprising receiving depth data, locating a person in the depth data, and mapping a physical space in front of the person to a screen space of a display device. The method further comprises forming an image of an avatar representing the person, outputting to a display an image of a user interface comprising an interactive user interface control, and outputting to the display device the image of the avatar such that the avatar faces the user interface control. The method further comprises detecting a motion of the person via the depth data, forming an animated representation of the avatar interacting with the user interface control based upon the motion of the person, and outputting the animated representation of the avatar interacting with the control.