VR Keyboard See-Through via Hand Gesture Activation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing keyboard see-through methods for virtual reality devices are complex and have poor compatibility, requiring additional computing power and affecting recognition accuracy due to various types of keyboards.
Innovation Solution
A method and apparatus that recognize hand actions of two hands to activate a keyboard see-through function, determining hand positions to display a physical keyboard in a specific area, using existing hand action recognition algorithms to reduce operational complexity and improve accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If keyboard recognition is performed using front-facing camera and recognition algorithm, then keyboard see-through function is achieved, but computing power consumption increases and recognition accuracy decreases due to numerous keyboard types and low compatibility
Solution Approach 1:
The patent extracts the keyboard recognition task from the VR device's front-facing camera and recognition algorithm, transferring it to the terminal device (mobile phone or computer). Only the necessary hand action recognition is performed by the VR device using its existing algorithms, while the terminal device handles the complex keyboard recognition and perspective calculation, thereby reducing the VR device's computing burden while maintaining compatibility across various keyboard types.
Solution Approach 2:
The patent introduces the terminal device as an intermediary between the VR device and the keyboard. The terminal device receives images from the VR device, performs keyboard recognition and perspective calculation, then returns the processed keyboard image to the VR device for display. This intermediary approach allows the VR device to leverage the terminal's superior computing capabilities without requiring enhanced hardware.
2Adaptability or versatility
If additional computing power is allocated for keyboard recognition, then recognition coverage is improved, but operational complexity and processing time increase
Solution Approach 1:
The patent extracts the computationally intensive keyboard recognition and perspective calculation tasks from the VR device and transfers them to the terminal device. This extraction allows the VR device to maintain real-time responsiveness while the terminal device handles the complex processing in the background, significantly reducing processing time without sacrificing recognition coverage.
Solution Approach 2:
The patent implements preliminary action by having the terminal device pre-process keyboard images and calculate perspective transformations before they are needed for display. The terminal device continuously monitors for keyboard detection, performs recognition and perspective calculation in advance, so that when the VR device needs to display the keyboard, the processing is already complete, eliminating delays.
3Ease of operation
If hand action recognition is used to activate keyboard see-through function, then operational complexity is reduced and compatibility is improved, but additional hand detection algorithms are required
Solution Approach 1:
The patent leverages the universal hand action recognition algorithms that are already built into most VR devices for other functions. By reusing these existing algorithms for keyboard activation, the system avoids adding new complexity while improving ease of operation. The same camera and processing pipeline used for general hand tracking are applied to detect keyboard-related hand gestures.
Solution Approach 2:
The patent implements self-service by utilizing the VR device's own existing hand recognition capabilities to activate the keyboard see-through function. The device serves itself by using its built-in algorithms to detect hand actions and trigger the appropriate function, eliminating the need for external activation methods or additional specialized detection algorithms.
Data Source
AI summary
The present application discloses a keyboard perspective method and apparatus for a virtual reality device, and the virtual reality device. The method comprises: recognizing hand actions of two hands of a user; if the hand actions of the two hands of the user meet a preset activation action, activating a keyboard perspective function of a virtual reality device; recognizing hand positions of the two hands of the user under the keyboard perspective function; and determining a keyboard perspective display area according to the hand positions of the two hands of the user so as to display a physical keyboard in a real scene in the keyboard perspective display area. Compared with traditional keyboard perspective schemes, use experience of the user is greatly improved.