Gesture Recognition Using Bayesian Networks for Speech Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current mobile computing devices face challenges in speech detection due to small input devices, difficulty in distinguishing speech from background noise, and issues with initiating speech recording at the right time, leading to user frustration.
Innovation Solution
A method using Bayesian networks, such as Hidden Markov Models, to identify gestures performed on a mobile device, allowing for accurate segmentation of accelerometer data to account for speed and determine appropriate actions, such as starting or stopping speech recording based on device pose.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If speech recording is used to interact with mobile devices, then ease of operation is improved, but reliability deteriorates due to difficulty in distinguishing speech from background noise
Solution Approach 1:
The patent combines multiple sensing modalities (accelerometer data, microphone data, gyroscope data) into a unified gesture recognition system. By merging data from these different sensors, the system can distinguish between intentional user gestures and environmental noise more reliably than speech recognition alone, thereby improving both ease of operation and reliability
Solution Approach 2:
The patent introduces gesture recognition as an intermediary mechanism between the user and speech recording. Instead of directly relying on speech detection, the system uses gesture detection as an intermediate step to trigger and control speech recording, providing a more reliable way to initiate and manage recording in noisy environments
2Productivity
If speech recording is initiated automatically, then productivity is improved, but reliability deteriorates due to difficulty in capturing user speech at the right time
Solution Approach 1:
The patent uses accelerometer data to detect preliminary gestures that indicate the user's intent to speak. By detecting these preliminary actions (such as bringing the device to the mouth), the system can proactively prepare for speech recording and initiate it at the optimal moment, improving both productivity and reliability
Solution Approach 2:
The patent implements feedback mechanisms where the system continuously monitors sensor data and adjusts recording initiation timing based on detected gesture patterns. This feedback loop allows the system to learn from user behavior and improve its timing for starting and stopping recording, ensuring speech is captured at the right time
3Measurement precision
If gesture recognition uses accelerometer time series data, then measurement precision is improved, but device complexity increases due to need for Bayesian networks and segmentation
Solution Approach 1:
The patent segments accelerometer time series data into distinct gesture phases (e.g., approach, contact, release) and processes each segment separately using Bayesian networks. This segmentation approach improves measurement precision by focusing analysis on specific gesture characteristics while managing complexity through modular processing of divided data segments
Data Source
AI summary
The disclosure generally relates to identifying gestures on mobile devices for use in connection with speech detection. A gesture may be identified using at least one Bayesian network, such as a Hidden Markov Model. Each Bayesian network corresponds to one of a plurality of different gesture types, which may include different gestures or similar gestures performed at different speeds, for example. Each Bayesian network includes states corresponding to partitioned segments in an accelerometer time series for the associated gesture type. Segmenting the accelerometer timeseries allows the Bayesian network to account for the speed at which the user makes the gesture, minimizing any effect of the user's speed in performing the gesture on identifying the gesture type. An action may be taken based on the identified gesture type. For example, speech recording may start or stop and/or a notification regarding speech recording may be provided depending on the gesture identified.