AI Text-to-Speech Error Correction Using Psychoacoustic Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing text-to-speech systems produce synthesized speech with inaccuracies and imperfections due to the complex interaction between mechanical and perceptual aspects of sound processing, which current neural networks fail to adequately address.
Innovation Solution
A neural network-based text-to-speech system is organized into a front-end subsystem for text analysis and a back-end subsystem for signal generation, using a smaller neural network to correct imperfections by applying error signals from a pre-existing knowledgebase, and employing psychoacoustic principles to train the network, reducing complexity and improving speech quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If traditional neural networks are used to synthesize speech from text, then speech generation capability is achieved, but the synthesized speech contains inaccuracies and imperfections that are perceptible to listeners
Solution Approach 1:
The speech synthesis task is segmented into two distinct components: a pre-existing knowledgebase that generates baseline speech from text input, and a neural network that specifically processes and corrects perceptual errors in the generated speech. This segmentation allows each component to focus on its specialized function, improving overall accuracy without requiring the entire system to be overly complex.
Solution Approach 2:
The neural network acts as an intermediary between the pre-existing knowledgebase and the final speech output. It receives the baseline speech signal, applies psychoacoustic error detection to identify perceptual inaccuracies, and generates corrective signals that are combined with the original speech to produce improved output.
2Reliability
If large and complex neural networks are deployed to capture speech generation directly from sound samples, then comprehensive speech synthesis is achieved, but processing requirements and computational resources increase significantly
Solution Approach 1:
Instead of training a neural network to perform the complete speech synthesis task from scratch, the system applies partial action by having the neural network focus only on the specific subtask of detecting and correcting perceptual errors in speech generated by the pre-existing knowledgebase. This reduces the computational burden while maintaining speech quality.
Solution Approach 2:
The system implements a feedback mechanism where the neural network analyzes the speech output from the pre-existing knowledgebase, identifies perceptual errors using psychoacoustic principles, and generates corrective feedback signals that are applied to improve the speech output. This iterative feedback process enhances speech quality without requiring excessive processing power.
3Manufacturing precision
If psychoacoustic principles are applied to train the neural network, then perceived speech quality improves, but training complexity increases
Solution Approach 1:
The training process utilizes psychoacoustic parameters such as perceived loudness, frequency sensitivity, and masking thresholds to define the error metric. By changing the optimization parameter from raw acoustic differences to psychoacoustically-weighted perceptual differences, the system trains the neural network to minimize errors that are actually audible to listeners, thereby improving perceived speech quality.
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
A technique improves training and speech quality of a text-to-speech (TTS) system having an artificial intelligence, such as a neural network. The TTS system is organized as a front-end subsystem and a back-end subsystem. The front-end subsystem is configured to provide analysis and conversion of text into input vectors, each having at least a base frequency, f0, a phenome duration, and a phoneme sequence that is processed by a signal generation unit of the back-end subsystem. The signal generation unit includes the neural network interacting with a pre-existing knowledgebase of phenomes to generate audible speech from the input vectors. The technique applies an error signal from the neural network to correct imperfections of the pre-existing knowledgebase of phenomes to generate audible speech signals. Speech signal specific modelling techniques in combination with applied psychoacoustic principles drive training efficiency of neural networks with positive impact on quality of generated speech signals.