Text-to-Speech Sub-Encoder and Decoder for Speaker Characteristic Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
End-to-end speech recognition models require large amounts of training data, and existing methods for generating speech signals from text-only data are inefficient, as they lack effective methods to capture speaker-specific utterance characteristics.
Innovation Solution
A processor-implemented text-to-speech method using a sub-encoder to determine a first feature vector from a partial section of a speech signal and an autoregressive decoder to generate a second speech signal based on context information, with the sub-encoder and decoder trained to reduce the loss between the feature vectors of the original and generated speech signals, employing a generative adversarial network for improved accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If text-to-speech models are trained using only text-only data without speech signals, then training data quantity can be increased, but the ability to capture speaker-specific utterance characteristics deteriorates
Solution Approach 1:
The method extracts speaker characteristics from speech signals in advance and stores them as embedding vectors before the main text-to-speech generation process. This preliminary extraction of speaker features allows the model to efficiently reuse speaker characteristics across multiple text conversions without requiring full speech signals each time, thus enabling both data efficiency and speaker accuracy.
Solution Approach 2:
The patent introduces speaker embedding vectors as an intermediary representation that bridges text-only data and speaker-specific characteristics. These embedding vectors serve as a compact mediator that captures essential speaker traits without requiring the full speech signal, allowing the model to generate speaker-specific speech from text alone while maintaining characteristic accuracy.
2Reliability
If end-to-end speech recognition models use large amounts of training data, then model performance improves, but data acquisition cost and processing time increase
Solution Approach 1:
The method creates compact copies of speaker characteristics in the form of embedding vectors that can be stored and reused. Instead of requiring the original speech signals for each conversion, the system uses these copied speaker feature representations, dramatically reducing data processing requirements while maintaining model performance.
Solution Approach 2:
The patent transforms the high-dimensional speech signal data into compressed embedding vector representations, changing the parameter dimensionality from full speech waveforms to condensed feature vectors. This parameter transformation reduces processing time and data requirements while preserving the essential speaker characteristics needed for high-quality text-to-speech conversion.
Data Source
AI summary
A processor-implemented text-to-speech method includes determining, using a sub-encoder, a first feature vector indicating an utterance characteristic of a speaker from feature vectors of a plurality of frames extracted from a partial section of a first speech signal of the speaker, and determining, using an autoregressive decoder, into which the first feature vector is input as an initial value, from context information of the text, a second feature vector of a second speech signal in which a text is uttered according to the utterance characteristic.