Speech Noise Reduction Using Text Bottleneck Features
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current text information extraction methods in text-to-speech systems face challenges in real-time operation and alignment accuracy for noise reduction, and require large amounts of training data with corresponding text annotations, making it difficult to implement effectively in practical applications.
Innovation Solution
A speech processing method that combines speech features with text bottleneck features using a trained unidirectional LSTM model, allowing for efficient noise reduction without the need for real-time text information and separate training corpora, improving noise reduction performance and speech quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If text information extraction method in TTS is used to extract text information for noise reduction, then noise reduction can be achieved, but real-time operation is difficult and alignment accuracy is challenging
Solution Approach 1:
The patent extracts only the essential text bottleneck features from speech signals using a trained ASR model, rather than performing full text extraction and alignment. This extraction approach obtains sufficient information for noise reduction while avoiding the computational burden of complete text processing, enabling real-time operation.
Solution Approach 2:
The patent introduces text bottleneck features as an intermediary representation between the speech signal and the noise reduction process. These features serve as a compact mediator that captures essential linguistic information without requiring full text alignment, bridging the gap between speech input and noise reduction output in real-time.
2Reliability
If text information extraction method in TTS is used, then noise reduction can be achieved, but alignment accuracy between text and speech is difficult to achieve
Solution Approach 1:
The patent extracts text bottleneck features that capture essential linguistic information without requiring precise temporal alignment between text and speech. By taking out only the necessary feature representations, the system achieves effective noise reduction while avoiding the alignment accuracy problem.
Solution Approach 2:
The ASR model is pre-trained to extract text bottleneck features that inherently encode linguistic structure and semantics. This preliminary action of feature extraction prepares the data in a form that is directly useful for noise reduction without requiring subsequent alignment operations.
3Reliability
If text information extraction method is used for noise reduction, then noise reduction performance can be improved, but large amounts of training data with text annotations are required
Solution Approach 1:
The ASR model serves multiple functions: it is trained for speech recognition but its extracted text bottleneck features are reused for noise reduction tasks. This multi-functionality allows the same model and training data to benefit multiple applications, reducing the need for separate large-scale annotated datasets for noise reduction.
Solution Approach 2:
The system uses the ASR model's own internal representations (text bottleneck features) to perform noise reduction, rather than requiring external annotated datasets. The model serves itself by providing features that can be directly applied to improve its noise reduction capability without additional training data requirements.
Data Source
Figure 1A
Figure 1B~2
Figure 3~4
AI summary
A speech processing method and apparatus, an electronic device, and a computer-readable storage medium. The method comprises: obtaining speech information to be processed (S101); determining a first speech feature and a first text bottleneck feature according to the speech information to be processed (S 102); determining a first combined feature vector according to the first speech feature and the first text bottleneck feature (SI03); and inputting the first combined feature vector to a trained unidirectional long short-term memory model, performing speech processing on the first combined feature vector to obtain de-noised speech information, and sending the de-noised speech information to other electronic devices for display (S104).