Virtual Reality Gesture Input with Vibration-Based Trigger Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing virtual reality input methods using camera-based hand tracking suffer from low accuracy due to hand blocking and complex algorithms, leading to delayed and inaccurate gesture recognition.
Innovation Solution
An interaction system combining a head-mounted display device with a hand-mounted device equipped with a vibration sensor and/or pulse wave sensor, utilizing biological vibration and pulse wave signals to detect input triggers, and integrating motion tracking data with camera imagery for improved gesture recognition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If camera-based hand tracking is used to sense touch operations, then the system can capture hand images for gesture recognition, but accuracy of input recognition result deteriorates due to hand self-blocking
Solution Approach 1:
The patent introduces an intermediary sensor (vibration sensor or pulse wave sensor) worn on the user's hand or wrist to detect touch operations. This intermediary device directly senses the physical touch action through vibration or pulse wave changes, bypassing the need for complex camera-based visual recognition and hand pose estimation algorithms, thereby improving recognition accuracy while reducing algorithmic complexity
Solution Approach 2:
The patent replaces the optical-mechanical camera-based recognition system with a direct physical sensing mechanism using vibration sensors or pulse wave sensors. These sensors detect mechanical vibrations or blood pulse changes caused by touch actions, substituting complex image processing with simpler physical signal detection, thus reducing algorithm complexity while maintaining or improving accuracy
2Productivity
If camera-based hand tracking with complex recognition algorithms is used, then gesture detection can be implemented, but output delay increases due to algorithm processing time
Solution Approach 1:
The patent replaces computationally intensive image processing algorithms with direct physical signal detection using vibration or pulse wave sensors. These sensors generate immediate electrical signals in response to touch-induced physical changes, eliminating the time-consuming steps of image capture, processing, and analysis required by camera-based systems, thus significantly reducing output delay
Solution Approach 2:
The patent changes the detection parameter from visual image data to direct physical signals (vibration frequency or pulse wave amplitude). This parameter change enables real-time detection since physical sensors respond instantaneously to touch actions, whereas camera systems require multiple frames and complex processing, thereby improving gesture input speed and reducing latency
3Measurement precision
If multiple frames of consecutive hand images are processed for gesture recognition, then spatial position tracking can be achieved, but processing complexity and time consumption increase
Solution Approach 1:
The patent extracts the essential detection function from the complex multi-frame image processing system by using a simple sensor that directly detects touch actions through vibration or pulse wave changes. This extraction eliminates the need for continuous image capture and processing, focusing only on the critical moment of touch interaction, thereby reducing processing time while maintaining detection accuracy
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enhances gesture input accuracy and reduces output delay by leveraging higher sampling frequencies and simpler algorithms, providing a more precise and real-time interaction experience.
Implementation Method 1
The target sensor signal includes a biological vibration wave signal detected by the vibration sensor
Implementation Method 2
a pulse wave signal detected by the pulse wave sensor
Data Source
Figure 1
Figure 2
Figure 3
AI summary
This application provides an interaction method, an electronic device, and an interaction system, and relates to the field of virtual reality technologies. The method includes: A head-mounted display device displays a virtual keyboard. In response to an input trigger operation of a user, a hand-mounted device sends hand detection data to the head-mounted display device based on a detected target sensor signal. After determining, based on the hand detection data, that an input trigger event occurs, the head-mounted display device determines an initial tap position of a target finger relative to the virtual keyboard based on a hand image captured by a camera. The hand-mounted device sends hand motion tracking data collected by a motion sensor to the head-mounted display device. In response to an input completion operation of the user, the head-mounted display device determines and displays an input result based on the initial tap position and the hand motion tracking data. According to the technical solutions provided in this application, accuracy of a gesture input in a scenario of virtual reality can be improved.