Source-Specific Speech Interactions Using Segmented Microphone Arrays
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In environments with high levels of non-voice interference, existing speech recognition systems face difficulties in accurately recognizing and understanding user speech, particularly when using far-field microphones that capture omnidirectional sounds.
Innovation Solution
A speech-based system employing both far-field and near-field microphones, where far-field microphones capture omnidirectional sounds and near-field microphones capture directional sounds, with separate ASR and NLU models optimized for each environment, and different TTS models for speech generation based on the source of audio input.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If far-field microphones are used to capture omnidirectional sounds, then the system can receive user speech from any position, but non-voice interference from the environment is also captured
Solution Approach 1:
The system segments the audio capture function into two distinct microphone types: far-field microphones for omnidirectional coverage and near-field microphones for directional capture. This segmentation allows the system to handle different spatial scenarios separately, using the appropriate microphone type based on the user's position and environmental noise levels.
Solution Approach 2:
Different microphone types are deployed with different spatial characteristics: far-field microphones provide omnidirectional coverage for far-distance capture, while near-field microphones provide directional capture for close-proximity speech. The system applies local quality by matching the appropriate microphone type to the specific spatial context of each interaction.
2Device complexity
If a single ASR model is used for all audio inputs, then the system structure is simpler, but recognition accuracy decreases in noisy environments
Solution Approach 1:
The ASR model is segmented into two specialized models: a far-field ASR model trained on audio from far-field microphones and a near-field ASR model trained on audio from near-field microphones. Each model is optimized for its specific audio characteristics, improving recognition accuracy by matching the model to the audio source type rather than using a single general-purpose model.
Solution Approach 2:
The system changes the parameters of the ASR model based on the audio source: different acoustic models, noise profiles, and training data characteristics are applied depending on whether the input comes from a far-field or near-field microphone. This parameter adaptation allows each model to be optimized for its specific operational context.
3Reliability
If different TTS models are used based on audio source, then speech generation is optimized for the environment, but system complexity increases
Solution Approach 1:
The TTS model selection is made dynamic based on the audio source characteristics. The system automatically selects the appropriate TTS model (far-field or near-field) based on which microphone captured the user's speech, allowing the speech generation to adapt to the environmental context without requiring manual configuration.
Solution Approach 2:
The system creates separate TTS models that mirror the characteristics of their corresponding ASR models and microphone types. This copying approach ensures consistency between the audio capture and speech generation characteristics, with each TTS model optimized to match the specific environmental conditions of its paired microphone type.
Data Source
AI summary
A speech system may be configured to operate in conjunction with a stationary base device and a handheld remote device to receive voice commands from a user. Voice commands may be directed either to the base device or to the handheld device. When performing automatic speech recognition (ASR), natural language understanding (NLU), dialog management, text-to-speech (TTS) conversion, and other speech-related tasks, the system may utilize various models, including ASR models, NLU models, dialog models, and TTS models. Different models may be used depending on whether the user has chosen to speak into the base device or the handheld audio device. The different models may be designed to accommodate the different characteristics of audio and speech that are present in audio provided by the two different components and the different characteristics of the environmental situation of the user.