Multi-Device Audio Attention for Reliable Microphone Signal Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems for selecting and using sensor data, such as microphone signals from multiple devices, lack the ability to dynamically assess the reliability of these signals, leading to inconsistent performance due to varying acoustic conditions and noise sources, which affects tasks like wakeword detection and speech recognition.
Innovation Solution
Implementing an attention-based neural network to estimate the reliability of sensor data from multiple devices by integrating input embedding vectors and generating device-wise context vectors, allowing for dynamic and data-driven control of audio devices based on the reliability of microphone signals.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional machine learning approaches are used for speech and audio analytics, then device complexity is reduced, but measurement precision and reliability of analytics outcomes deteriorate
Solution Approach 1:
The system segments the analytics process into multiple independent channels (e.g., pitch channel, volume channel, pause channel) that can be processed separately and then integrated. Each channel applies specific ML models tailored to its characteristics, improving precision without requiring a single complex monolithic system.
Solution Approach 2:
The patent introduces the dimension of multi-channel processing by analyzing speech signals through multiple independent analytical channels simultaneously. This dimensional expansion allows parallel processing of different speech characteristics, achieving higher overall precision while maintaining manageable complexity through modular architecture.
2Reliability
If multiple ML models are applied to different channels, then analytics reliability is improved, but device complexity increases
Solution Approach 1:
The system employs a universal multi-channel architecture where each channel serves a specific analytical function but follows a consistent processing framework. This universality allows the system to handle multiple speech characteristics reliably while maintaining a standardized structure that prevents complexity from escalating.
Solution Approach 2:
By dividing the analytics system into segmented channels, each with its own specialized ML model, the system achieves reliable analytics for each speech characteristic independently. The segmentation allows complexity to be distributed and managed at the channel level rather than concentrated in a single complex system.
3Loss of information
If comprehensive speech signal analysis is performed, then information completeness is improved, but processing time increases
Solution Approach 1:
The multi-channel approach adds dimensional parallelism to the processing architecture, allowing comprehensive analysis of multiple speech characteristics to occur simultaneously rather than sequentially. This dimensional change enables complete information extraction without proportional increases in processing time.
Solution Approach 2:
The system maintains continuous useful action by processing multiple speech channels in parallel throughout the analysis period. All channels operate simultaneously and continuously, ensuring no information is lost to sequential processing delays while maintaining efficient utilization of computational resources.
Data Source
Figure 1A
Figure 1B
Figure 2
AI summary
Some disclosed methods involve receiving sensor data, including microphone data, from each of a plurality of devices in the environment, producing an input embedding vector corresponding to each sensor, producing a device-wise context vector corresponding to each device, obtaining ground truth data and comparing each device-wise context vector with the ground truth data, to produce a comparison result. The comparing may involve an attention-based process. Some disclosed methods involve generating one or more current output analytics tokens based, at least in part, on the comparison result and controlling the operation of at least one device based, at least in part, in the one or more current output analytics tokens. The controlling may involve controlling at least one of a speaker operation or a microphone operation.