On-Body Vibration Gesture Recognition Without User Calibration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing wearable devices, such as smartwatches, face challenges with small screens leading to tedious and error-prone interactions, and existing solutions like speech recognition are noisy or require user calibration, which is inconvenient and impractical.
Innovation Solution
A vibration-based interaction system using a single IMU sensor in smartwatches employs an unsupervised Siamese adversarial learning method to adapt to user variations, enabling robust finger interactions without requiring user labeling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Volume of moving object
If a small screen is used in wearable devices, then the device size is reduced, but interaction becomes tedious and error-prone
Solution Approach 1:
The patent replaces traditional mechanical touchscreen interaction with vibration-based gesture recognition. The IMU sensor detects vibration patterns from finger taps and gestures on the wrist, converting mechanical body movements into digital signals for device control, thereby eliminating the need for a large touchscreen while improving interaction ease.
2Ease of operation
If speech recognition is used for interaction, then hands-free operation is enabled, but the system becomes noisy and requires user calibration
Solution Approach 1:
The patent substitutes acoustic speech recognition with mechanical vibration sensing. The IMU sensor detects physical vibration patterns from finger gestures on the wrist, providing a noise-immune alternative to speech-based systems while maintaining hands-free operation capability.
Solution Approach 2:
The system automatically adapts to individual user gestures through unsupervised Siamese adversarial learning, eliminating the need for manual calibration. The model learns user-specific vibration patterns autonomously during normal device usage, making the interaction system self-adjusting and user-specific.
3Measurement precision
If user calibration is required for accurate gesture recognition, then measurement precision is improved, but the system becomes inconvenient and impractical
Solution Approach 1:
The system performs automatic self-calibration through unsupervised learning. The Siamese adversarial network continuously learns user-specific vibration patterns from unlabeled gesture data during normal usage, eliminating the need for manual calibration procedures while maintaining high recognition accuracy.
Solution Approach 2:
The model performs preliminary adaptation to user gestures automatically during initial device setup and ongoing usage. Through continuous unsupervised learning, the system pre-adapts to individual user patterns before specific interaction tasks are performed, ensuring high accuracy without requiring user intervention.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The system achieves high accuracy (97%) in real-time finger interaction, is resistant to deployment variations, and provides a user-friendly experience by adapting to individual user changes without supervised calibration.
Implementation Method 1
a vibration sensor to detect a vibration event
Data Source
AI summary
Disclosed are various embodiments for recognition of on body touch interactions and gestures using an on-body device. A sample of vibration data from the vibration sensor is input into a trained convolutional neural network. The vibration data is generated from a vibration event. In response, the trained convolutional neural network outputs one of a plurality of predefined vibration event descriptors. The trained convolutional neural network is adapted based at least in part on a plurality of Siamese contrastive loss calculations. Each Siamese contrastive loss calculation is generated from a corresponding pair of preexisting samples of vibration data from a pool of preexisting samples of vibration data.