Speech Recognition Using User Time Markers in Noisy Audio
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automated speech recognition (ASR) systems struggle with robustness in the presence of background noise and competing speech, leading to inaccurate transcriptions due to the difficulty in maintaining multiple task-specific models and the ill-posed nature of speaker separation in noisy environments.
Innovation Solution
The use of a sequence of time markers input by the user in cadence with their speech, correlated with the audio signal to enhance speech features, allowing the ASR system to separate target speech from background noise and competing speech, using techniques like blind speaker diarization and point-by-point time difference matching.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional ASR systems are used without time markers, then the system complexity remains low, but the transcription accuracy deteriorates in noisy environments
Solution Approach 1:
The system performs preliminary action by having users input time markers before the actual speech recognition process. These time markers are collected and stored in advance, then used during transcription to guide the ASR system in identifying speech segments and separating target speech from background noise, thereby improving accuracy without requiring complex post-processing
2Reliability
If multiple task-specific models are maintained to handle different noise conditions, then the robustness improves, but the device complexity increases
Solution Approach 1:
The system applies universality by creating a single ASR model that can handle multiple noise conditions through the use of time markers. Instead of maintaining separate models for different noise scenarios, the universal model uses the provided time markers to adaptively identify and process target speech segments, making one model serve multiple functions across various acoustic environments
3Measurement precision
If speaker separation techniques are applied to separate target speech from background noise, then the transcription accuracy improves, but the difficulty of detecting and measuring increases
Solution Approach 1:
The system uses time markers as an intermediary element to facilitate speaker separation. Rather than directly attempting to separate speakers through complex acoustic analysis, the time markers serve as a mediator that provides temporal boundaries and guidance to the ASR system, making the separation task more manageable and accurate by indicating where target speech occurs
Data Source
AI summary
A method for separating target speech from background noise contained in an input audio signal includes receiving the input audio signal captured by a user device, wherein the input audio signal corresponds to target speech of multiple words spoken by a target user and containing background noise in the presence of the user device while the target user spoke the multiple words in the target speech. The method also includes receiving a sequence of time markers input by the target user in cadence with the target user speaking the multiple words in the target speech, and correlating the sequence of time markers with the input audio signal to generate enhanced audio features that separate the target speech from the background noise in the input audio signal. The method also includes processing, using a speech recognition model, the enhanced audio features to generate a transcription of the target speech.