Speech Recognition with User Time Markers in Background Noise
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automatic speech recognition (ASR) systems face challenges in accurately transcribing speech in noisy environments due to background noise and competing speech, with current models struggling to effectively separate and recognize target speech from these interferences.
Innovation Solution
The use of a sequence of time markers input by the user in cadence with their speech, correlated with the audio signal to enhance audio features, allowing the ASR system to separate target speech from background noise and improve transcription accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If neural network-based end-to-end models and large-scale training data are used to improve ASR robustness, then speech recognition accuracy improves, but performance deteriorates in harsh background noise and competing speech conditions
Solution Approach 1:
The patent segments the audio signal into multiple frequency sub-bands using filter banks. Each sub-band is processed independently by the speech recognition model, allowing the system to focus on specific frequency ranges where target speech is present while ignoring bands dominated by background noise or competing speech. This segmentation approach enables the system to maintain accuracy even when overall noise levels are high.
Solution Approach 2:
The patent applies different processing characteristics to different frequency sub-bands. By analyzing spectral features and identifying which sub-bands contain target speech versus background noise, the system adjusts processing parameters locally for each band. This allows optimal recognition performance in clean bands while suppressing or ignoring noisy bands, thereby improving reliability in noisy conditions without sacrificing accuracy in clean conditions.
2Productivity
If the ASR system processes the entire audio signal as a whole, then computational efficiency is maintained, but the system cannot effectively separate target speech from background noise and competing speech
Solution Approach 1:
The audio signal is divided into frequency sub-bands using filter banks, creating parallel processing channels. Each sub-band is processed independently through the speech recognition model, enabling effective separation of target speech from background noise and competing speech. This segmentation maintains computational efficiency by using simple filter bank operations rather than complex separation algorithms, while still achieving accurate speech separation through the frequency-domain decomposition.
3Device complexity
If traditional speech recognition models are used, then device complexity remains low, but the system struggles to separate target speech from competing speech in noisy environments
Solution Approach 1:
The system uses relatively simple filter bank operations to segment the audio into frequency sub-bands, which can be implemented with low computational overhead. Each sub-band is then processed by the speech recognition model, which makes decisions based on localized spectral information. This approach improves reliability in noisy conditions by focusing on relevant frequency regions while maintaining manageable device complexity through the use of standard filter bank technology and existing speech recognition models.
Solution Approach 2:
The patent transforms the speech recognition problem from the time domain to the frequency domain by applying filter banks. This dimensional transformation allows the system to exploit spectral information that is not apparent in the time domain, enabling better separation of target speech from background noise and competing speech. The frequency-domain representation provides an additional dimension for discrimination without requiring significantly more complex processing.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method (300) includes receiving an input audio signal (202) captured by a user device (110), wherein the input audio signal corresponds to target speech (12) of multiple words spoken by a target user (10) and containing background noise in the presence of the user device while the target user spoke the multiple words in the target speech. The method also includes receiving a sequence of time markers (204) input by the target user in cadence with the target user speaking the multiple words in the target speech, and correlating the sequence of time markers with the input audio signal to generate enhanced audio features (145) that separate the target speech from the background noise in the input audio signal. The method also includes processing, using a speech recognition model (160), the enhanced audio features to generate a transcription (165) of the target speech.