Speech Generator Using ASR and Neural Network Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Smart speakers often generate synthesized speech that sounds unnatural and struggles to accurately identify user speech due to poor quality or low intelligibility, negatively impacting the user experience.
Innovation Solution
A speech generator system that includes signal processing components for automatic speech recognition (ASR) and speech state estimation using neural networks to produce a synthesized audio signal that mimics the tone, emotion, and context of the original speech, improving the naturalness and intelligibility of the output.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If traditional text-to-speech (TTS) conversion is used to generate synthesized speech, then the smart speaker can convert query results into audio output, but the synthesized speech sounds unnatural and different from the user's normal speech
Solution Approach 1:
The speech generation process is divided into multiple stages: extracting acoustic features from the user's original speech, generating text from the speech via ASR, synthesizing speech from the text using TTS, and then refining the synthesized speech by re-applying the extracted acoustic features. This segmentation allows each stage to focus on specific aspects of speech generation, improving overall naturalness.
Solution Approach 2:
The system uses the user's original speech as a reference to extract acoustic features (such as pitch, tone, and timbre) and applies these features as feedback to the synthesized speech generation process. This feedback loop ensures the synthesized speech matches the user's natural speech characteristics rather than sounding like generic machine speech.
2Reliability
If the smart speaker uses standard speech recognition, then it can process user queries, but it struggles to accurately identify user speech due to poor quality or low intelligibility
Solution Approach 1:
The system performs preliminary speech enhancement and feature extraction before the main speech recognition process. By pre-processing the speech signal to extract and enhance acoustic features, the system prepares the speech data in a form that is more suitable for accurate recognition, thereby improving reliability even when input quality is poor.
Solution Approach 2:
The system introduces an intermediary speech enhancement stage between the microphones and the speech recognition engine. This intermediary process includes noise reduction, echo cancellation, and feature extraction that bridges the gap between raw, potentially poor-quality speech and the requirements of the recognition system, improving overall accuracy.
Data Source
AI summary
In a particular aspect, a speech generator includes a signal input configured to receive a first audio signal. The speech generator also includes at least one speech signal processor configured to generate a second audio signal based on information associated with the first audio signal and based further on automatic speech recognition (ASR) data associated with the first audio signal.