Gesture-Tracked Virtual Keyboard for Environment-Independent Text Input
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing virtual reality and augmented reality technologies limit the generation of virtual objects to specific physical environments and complicate text input operations, making them unintuitive and cumbersome.
Innovation Solution
An input apparatus and method that uses a camera and processor to capture hand gestures, generating a virtual keyboard on a virtual plane based on palm position and recognizing typing and editing gestures to input commands intuitively without reliance on physical planes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If virtual reality and augmented reality technology is used to generate virtual objects, then the ability to create virtual content is improved, but the operation becomes limited to specific physical environments and requires reference objects
Solution Approach 1:
The patent replaces the mechanical system of physical keyboards and touchscreens with a gesture recognition system using computer vision. The camera captures hand movements and the processor translates these into input commands, eliminating the need for physical contact surfaces and enabling operation in any environment where gestures can be performed.
Solution Approach 2:
The patent introduces a virtual plane as an intermediary between the user's gestures and the digital interface. This virtual plane, generated in augmented or virtual reality space, serves as the interaction surface that responds to hand gestures, allowing users to interact with digital content without physical constraints.
2Loss of information
If traditional text input methods are used in virtual reality and augmented reality, then text can be input, but the operation becomes complicated and unintuitive
Solution Approach 1:
The patent creates a virtual keyboard that is a visual copy of a physical keyboard layout in the virtual space. Users perform gestures that mimic physical typing actions, and the system translates these gestures into text input commands, preserving the intuitive nature of keyboard interaction while removing physical constraints.
Solution Approach 2:
The patent replaces the mechanical interaction of pressing physical keys with gesture recognition. The camera captures hand movements and the processor interprets these gestures as typing actions, substituting the mechanical keyboard system with an optical recognition system that maintains the familiar typing paradigm.
3Ease of operation
If a virtual keyboard is generated on a virtual plane based on palm position, then environment-independent interaction is achieved, but the system complexity increases
Solution Approach 1:
The patent creates a universal interaction system where the same gesture recognition and virtual plane generation mechanisms work across different environments and applications. The camera and processor combination serves multiple functions: capturing gestures, generating the virtual plane, tracking hand movements, and translating gestures into input commands, providing a unified interface for various tasks.
Data Source
AI summary
An input apparatus is configured to execute the following operations. A first gesture of a user is determined based on first hand images of hand images. In response to the first gesture matching an activating gesture, a virtual keyboard is generated on a virtual plane at a first time point, and the virtual plane is generated based on a palm position corresponding to the first gesture. A second gesture of the user is determined based on second hand images corresponding to a second time point of the hand images, and the first time point is earlier than the second time point. In response to the second gesture matching a typing gesture, an input command corresponding to the typing gesture is generated based on a movement between the second gesture and the virtual keyboard.