Acoustic Hand Motion Tracking for Smart Device Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current smart devices, such as smart speakers, rely on voice commands for control, which are ineffective in noisy environments and cumbersome for tasks like screen interaction or menu selection, and lack a means for motion-based control using hand or body motion.
Innovation Solution
A method using acoustic signals received by microphones to track the movement of objects, transforming signals into two-dimensional sinusoids to estimate angle-of-arrival and propagation distance, enabling the determination of object location and performing commands based on hand motion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If voice commands are used to control smart devices, then the device can be operated hands-free, but the control degrades significantly in noisy environments due to interference
Solution Approach 1:
The patent introduces an intermediary acoustic signal transmission mechanism between the user and the smart device. Instead of relying on voice commands that are vulnerable to noise interference, the system uses acoustic signals transmitted through a medium (such as a table or surface) that serves as a mediator to carry the control information to the device's microphones, thereby improving reliability in noisy environments while maintaining hands-free operation
Solution Approach 2:
The patent replaces the acoustic field-based voice command system with a mechanical vibration-based control system. By using vibrations transmitted through solid objects (tables, surfaces) to carry control information, the system substitutes the vulnerable acoustic wave transmission in air with more stable mechanical vibration transmission through solids, thereby improving control reliability in noisy environments
2Ease of operation
If voice-based control is used for screen interaction or menu selection, then the device can be controlled without contact, but the interaction becomes cumbersome
Solution Approach 1:
The patent segments the control interface into distinct acoustic signal zones corresponding to different screen elements or menu options. By dividing the acoustic space into segmented regions, each representing a specific control target, the system enables precise selection of screen elements through directional acoustic signals, reducing interaction complexity while maintaining contactless operation
Solution Approach 2:
The patent adds a spatial dimension to voice-based control by incorporating directional and positional information into the acoustic signal processing. Instead of treating all voice commands equally, the system analyzes the spatial characteristics (angle of arrival, distance) of acoustic signals to map them to specific screen elements, thereby simplifying menu selection and screen interaction through intuitive spatial gestures
3Measurement precision
If motion-based control using hand motion is implemented, then the control precision for screen interaction is improved, but the device complexity increases due to signal processing requirements
Solution Approach 1:
The patent utilizes mechanical vibration analysis of acoustic signals to detect and track hand motions. By analyzing the vibration characteristics (frequency, amplitude, patterns) of acoustic signals reflected from or emitted by the user's hand, the system achieves precise motion tracking without requiring complex processing algorithms, thereby improving measurement precision while controlling device complexity
Solution Approach 2:
The patent enables the acoustic signal processing system to automatically identify and track hand motions without requiring explicit programming for each motion type. The system self-adapts by analyzing the inherent characteristics of acoustic signals from moving hands and automatically adjusting tracking parameters, reducing the complexity of signal processing while maintaining high measurement precision
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables accurate and reliable room-scale motion tracking and control of smart devices, improving usability in noisy environments and supporting multiple languages, while complementing voice-based control.
Implementation Method 1
receiving acoustic signals using one or more microphones in the device transmitted from an object of a user
Implementation Method 2
transforming at least a portion of the received acoustic signals into two-dimensional sinusoids whose frequencies are proportional to an angle-of-arrival and a propagation distance of the object
Data Source
AI summary
A method, device and computer program product for controlling the device by tracking a movement of a hand or other objects. The device receives acoustic signals. At least a portion of the received signals are transformed into two-dimensional sinusoids whose frequencies are proportional to an angle-of-arrival (AoA) and a propagation distance of the reflected signals. An AoA-distance profile is derived based on signals received from the object by evaluating frequencies of the two-dimensional sinusoids. Then, an AoA-distance pair is derived from the AoA-distance profile. A current location of the object is determined based on the estimated AoA-distance pair. The device then performs a command in response to detecting that the user moved to perform the command based on prior and current locations of the object.