Neural Network Speech Segmentation for Local TTS
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Intelligent automated assistants face limitations in generating realistic speech outputs locally on user devices due to processing capabilities and latency issues in communication with remote servers, resulting in less efficient and less realistic speech delivery.
Innovation Solution
The method involves using multiple neural networks to generate speech by first creating segments of a spectrogram from text, which are then processed to produce more realistic speech segments, enhancing processing efficiency and reducing power consumption on user devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If speech is generated locally on the electronic device, then processing speed is improved, but speech realism deteriorates
Solution Approach 1:
The patent segments the speech generation process into multiple stages using different neural networks: a first neural network generates initial speech segments locally for speed, while a second neural network refines these segments to improve realism. This segmentation allows each network to specialize in specific aspects of speech generation.
Solution Approach 2:
The patent merges the capabilities of multiple neural networks working together - the first network provides fast initial generation while the second network adds refinement for realism. This combination of multiple models achieves both speed and quality that neither model could achieve alone.
2Manufacturing precision
If speech is generated using a remote electronic device, then speech realism is improved, but communication latency increases
Solution Approach 1:
The patent divides the speech generation task into segments that can be processed progressively. The first neural network generates initial segments quickly, and only the refinement segments are sent to the remote server, reducing the amount of data transmitted and minimizing latency while still achieving high realism.
Solution Approach 2:
The first neural network performs preliminary speech generation locally before any remote communication occurs. This preliminary action completes the basic speech generation function, and only the refinement step requires remote assistance, significantly reducing overall latency compared to generating all speech remotely.
3Manufacturing precision
If multiple neural networks are used to generate speech, then speech realism is improved, but device complexity increases
Solution Approach 1:
The patent segments the speech generation function across multiple neural networks with distinct roles: the first network handles initial generation and the second handles refinement. This functional segmentation allows each network to be optimized for its specific task, managing overall system complexity through specialization.
Solution Approach 2:
The system implements a multi-functional architecture where the first neural network can operate independently for basic speech generation needs, while the second neural network provides enhancement when available. This universal design allows the system to adapt between different operational modes (local-only, remote-only, or hybrid) to manage complexity.
Data Source
AI summary
Systems and processes for generating speech from text are provided. An example method of generating speech from text includes, at an electronic device having at least one processor and memory, obtaining text; generating a plurality of segments of a spectrogram using a first neural network, each spectrogram segment of the plurality of spectrogram segments representing a portion of the text; generating, based on the plurality of spectrogram segments, a plurality of speech segments using a second neural network; and providing the plurality of speech segments as a speech output.