3D Virtual Keyboard Text Input for AR Hand Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Augmented reality (AR) and virtual reality (VR) systems face challenges in accurately recognizing three-dimensional hand movements, leading to difficulties in text input due to the complexity of distinguishing between one and two-dimensional movements, resulting in mishandling failures when using virtual QWERTY keyboards with small character spaces.
Innovation Solution
A three-dimensional virtual keyboard system captures user movements along multiple axes, allowing the identification of characters and displaying proposed words using a dictionary database, enabling users to select desired words in a more accurate and intuitive manner within a three-dimensional virtual space.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a virtual QWERTY keyboard with small character spaces is used in AR/VR systems, then the text input function is provided, but the accuracy of recognizing three-dimensional hand movements deteriorates, leading to mishandling failures
Solution Approach 1:
The patent transitions from traditional 2D virtual keyboard interaction to 3D spatial interaction by mapping keyboard characters along multiple axes (x-axis for rows, y-axis for columns) in a three-dimensional virtual space. Users navigate through character positions using 3D hand movement along these axes, and proposed words are displayed along a z-axis, enabling accurate recognition of hand movements through multi-dimensional spatial positioning rather than relying on small 2D character spaces
2Quantity of substance
If the character space in a virtual keyboard is reduced to fit more characters, then the completeness of text input is improved, but the difficulty of detecting and measuring hand movements increases, resulting in recognition failures
Solution Approach 1:
Instead of compressing characters into smaller 2D spaces, the patent distributes characters along 3D axes where the x-axis represents keyboard rows and the y-axis represents columns. This spatial distribution across multiple dimensions increases the effective character space without requiring smaller individual character targets, making hand movement detection easier while accommodating more characters
Solution Approach 2:
The patent segments the text input process into distinct 3D navigation stages: first navigating along the x-axis to select a row, then along the y-axis to select a column, and finally along the z-axis to view and select proposed words. This segmentation of the input process into sequential 3D movements simplifies the detection of hand movements at each stage while maintaining access to the complete character set
Data Source
AI summary
A method, computer system, and computer program product for implementing three-dimensional text input in an augmented reality system are provided. The embodiment may include capturing a first user hand position along a first axis of a three-dimensional virtual space. The embodiment may also include identifying a first character along the first axis corresponding to the first user hand position. The embodiment may further include capturing a second user hand position along a second axis of the three-dimensional virtual space. The embodiment may also include identifying a second character along the second axis corresponding to the second user hand position. The embodiment may further include identifying one or more proposed words beginning, consecutively, with the identified first character and the identified second character using a dictionary database. The embodiment may also include displaying the one or more identified proposed words on a third axis of the three-dimensional virtual space.