Virtual Input Device UI Gesture Recognition and Gaze Reset
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing gesture recognition-based command input methods require complex technology and are not suitable for environments without a real input device, and they do not effectively support visually impaired users or maintain input consistency when the user looks away and returns.
Innovation Solution
A device and method for displaying a user interface of a virtual input device using motion recognition, which includes a display, storage for gesture templates, a camera for capturing user gestures, and a controller to generate and manage the UI, supporting voice guidance and sound feedback, and resetting the finger location when the user looks away.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If gesture recognition technology is used for command input, then input convenience is improved, but device complexity increases due to requiring additional voice recognition technology
Solution Approach 1:
The patent extracts and removes the voice recognition component from the gesture recognition system. The controller is configured to recognize gestures through camera-based motion detection alone, eliminating the need for integrated voice recognition technology and its associated complexity while maintaining gesture-based command input functionality.
Solution Approach 2:
The patent creates a virtual copy of a remote controller displayed on the screen that replicates the functionality of a physical remote. This virtual remote controller copy allows users to input commands through gesture recognition without needing actual physical input devices, simplifying the system architecture while maintaining input convenience.
2Adaptability or versatility
If virtual input device UI is displayed based on gesture recognition, then input method versatility is improved, but reliability decreases when user looks away from screen
Solution Approach 1:
The patent implements a feedback mechanism where the controller continuously monitors the user's gaze direction using the camera. When the user looks away from the screen, the system detects this through gaze analysis and provides feedback by resetting the finger location on the virtual remote controller to its initial position, ensuring reliable and consistent command input regardless of user attention state.
Solution Approach 2:
The system performs preliminary gaze detection and finger location tracking before the user actually inputs commands. By continuously monitoring gaze direction and pre-positioning the virtual finger location, the system ensures that when the user returns their gaze to the screen, the input state is already prepared and consistent, maintaining reliability across interruptions.
3Adaptability or versatility
If camera-based gesture recognition is implemented, then elimination of physical input devices is achieved, but measurement precision decreases for finger location tracking
Solution Approach 1:
The patent introduces an intermediary virtual remote controller displayed on the screen between the camera-based gesture detection and the command execution. The controller displays a visual representation of the user's finger location on the virtual remote, allowing for precise visual feedback and correction. This intermediary interface compensates for the lower precision of camera-based tracking by providing visual confirmation and enabling users to adjust their gestures accurately.
Data Source
AI summary
A display device includes a user interface (UI) generator configured to generate a UI of the virtual input device; a camera configured to photograph a user's gesture; and a controller configured to compare the photographed user's gesture with a user's gesture stored in the template and to control the UI generator to display a UI of a virtual input device corresponding to the matched user's gesture.