Blind Gesture Recognition for Bi-Manual UI Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing bi-manual input techniques require users to constantly shift visual attention between hands, disrupting the flow of activity and not optimizing operations for the non-dominant hand, which is often not designed for blind operation or fine motor control.
Innovation Solution
Implementing blind user interface elements that allow gestural input with one hand without shifting visual attention, where blind gestures can be recognized and associated with operations performed by the other hand, enabling blind control and manipulation of user interface elements regardless of scale, orientation, or location.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If conventional bi-manual input techniques are used, then both hands can manipulate user interface elements, but visual attention must constantly shift between hands disrupting the flow of activity
Solution Approach 1:
The patent divides the user interface into two distinct interaction modes: blind gestures for the non-dominant hand and visual gestures for the dominant hand. This segmentation allows each hand to operate independently without requiring visual attention shifts, resolving the contradiction between bi-manual capability and continuous visual focus.
Solution Approach 2:
The system introduces an intermediary mapping mechanism that translates blind gestures from the non-dominant hand into corresponding operations on the display. This intermediary layer enables the non-dominant hand to control UI elements without direct visual feedback, eliminating the need for attention shifts while maintaining operational capability.
2Measurement precision
If user interface elements are designed for fine motor control and close visual attention, then precise operations can be performed, but the non-dominant hand cannot operate blindly
Solution Approach 1:
The patent applies different interaction qualities to different hands: the dominant hand receives visual feedback and performs precision operations, while the non-dominant hand performs blind operations with coarser control. This local differentiation allows each hand to operate in its optimal mode without compromising overall system precision.
Solution Approach 2:
The system performs preliminary mapping of blind gesture zones to specific UI elements before the user executes the gesture. This pre-establishment of correspondence allows the non-dominant hand to operate blindly with confidence, knowing the gesture will affect the intended target, thereby enabling blind operation without sacrificing precision.
3Productivity
If the non-dominant hand is used for input, then bi-manual efficiency can be improved, but the hand lacks fine motor control and visual attention
Solution Approach 1:
The patent creates a virtual copy of the UI element's control interface that responds to blind gestures from the non-dominant hand. This copied interface maintains the functional capabilities needed for operation while being accessible through blind input, thereby enabling the non-dominant hand to perform efficient operations despite lacking fine motor control and visual attention.
Data Source
AI summary
Techniques for bi-manual control of an operation may allow gestural input for the operation to be inputted blindly with one hand while a user's attention remains on the other hand. An operation invoked and controlled by the gestural input may automatically associate with an operation controlled by the other hand. The gestural input may facilitate blind control of the operation by being able to be initially recognized without regard for the scale, orientation, and/or location of the invoking gestural input. Moreover, scale and/or orientation of the operation may be determined by the gestural input. Changes in scale and/or orientation of the gestural input during the blind operation may change the scale and/or orientation of the operation. The operation may manifest by displaying corresponding graphics at a location independent of the gestural input or by associating functionality of the blind operation with an operation performed by the other hand.