Ultrasonic Gesture Recognition via Beamforming and Deep Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current gesture recognition technologies for mobile interactive devices, such as head-mounted displays, face high energy costs due to illumination and processing complexities, limiting their use in mobile devices where energy efficiency is crucial.
Innovation Solution
An ultrasonic gesture recognition system that transmits ultrasonic chirps, collects return signals via a microphone array, generates acoustic images using beamforming, and classifies gestures using a deep learning system, specifically a convolutional neural network with a long short-term memory layer, to reduce energy consumption and increase gesture recognition capacity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If optical sensors are used for gesture recognition, then gesture recognition accuracy is improved, but energy consumption increases
Solution Approach 1:
The patent replaces optical sensing systems with ultrasonic acoustic sensing systems. The ultrasonic sensor transmits acoustic waves and receives reflections to detect gestures, substituting the optical illumination and detection mechanism with acoustic wave transmission and reception, thereby reducing energy consumption while maintaining gesture recognition capability
Solution Approach 2:
The patent changes the fundamental sensing parameter from optical wavelength to ultrasonic frequency. By operating in the acoustic domain (20 kHz to 200 kHz) rather than optical domain, the system achieves gesture recognition with lower energy requirements, as ultrasonic transmission and reception consumes less power than continuous optical illumination and image processing
2Reliability
If optical sensors with continuous illumination are used, then gesture detection capability is improved, but power consumption increases
Solution Approach 1:
The patent employs periodic ultrasonic pulse transmission instead of continuous illumination. The ultrasonic sensor transmits pulses at specific intervals and detects reflections during these periodic bursts, enabling gesture detection while consuming power only during transmission and reception windows rather than continuously, thus significantly reducing average power consumption
Solution Approach 2:
The ultrasonic system uses the ambient acoustic environment and natural sound reflections for detection. The sensor transmits ultrasonic waves that bounce off objects and return to the sensor, utilizing the existing acoustic properties of the environment without requiring additional active illumination or external energy sources, making the system energy-efficient
3Measurement precision
If beamforming and deep learning processing are applied, then gesture recognition accuracy is improved, but computational complexity increases
Solution Approach 1:
The patent performs beamforming processing on the raw ultrasonic signals before feeding them to the deep learning classifier. By pre-processing the signals to enhance spatial directionality and extract relevant acoustic features through beamforming, the system reduces the computational burden on the subsequent deep learning model, as the input data is already organized and enhanced with directional information
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The ultrasonic gesture recognition system significantly reduces energy consumption, enabling mobile devices to support nearly 100,000 gestures within the same energy budget as optical sensors, representing a 200-fold increase in gesture recognition capability.
Implementation Method 1
receive return signals of the ultrasonic signal that are reflected from the gesture
Implementation Method 2
generates an acoustic image with a beamformed frequency return signal for each of a plurality of directions
Data Source
AI summary
An ultrasonic gesture recognition system is provided that recognizes gestures based on analysis of return signals of an ultrasonic pulse that is reflected from a gesture. The system transmits an ultrasonic chirp and samples a microphone array at sample intervals to collect a return signal for each microphone. The system then applies a beamforming technique to frequency domain representations of the return signals to generate an acoustic image with a beamformed return signal for multiple directions. The system then generates a feature image from the acoustic images to identify, for example, distance or depth from the microphone array to the gesture for each direction. The system then submits the feature image to a deep learning system to classify the gesture.