Voice Processing With Neural Feature Generation for Natural Speech Editing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing speech editing methods rely heavily on a database of speech segments, leading to poor auditory perception when the database is insufficient, resulting in unnatural corrected speech with inconsistencies in timbre and rhythm.
Innovation Solution
A method that uses a neural network to generate a second speech feature based on the first speech feature of the original speech, ensuring the target edited speech matches the original speech in rhythm, timbre, and signal-to-noise ratio, thereby improving auditory perception.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If speech editing is performed by replacing erroneous segments with segments from a database, then content errors can be corrected, but auditory perception deteriorates when the database is insufficient
Solution Approach 1:
The patent changes the fundamental parameter of speech editing from database segment replacement to neural network-based feature generation. Instead of relying on pre-recorded segments, the system generates speech features (rhythm, timbre, signal-to-noise ratio) through neural network processing, fundamentally altering how corrected speech is produced and eliminating database dependency
Solution Approach 2:
The patent substitutes the mechanical database retrieval and segment matching system with a neural network-based intelligent generation system. The neural network processes the original speech and target text to directly generate appropriate speech features, replacing the manual/database-driven segment selection process with automated intelligent synthesis
2Productivity
If database segments are used for speech editing, then content can be corrected quickly, but timbre and rhythm consistency deteriorates
Solution Approach 1:
The system directly controls and generates key speech parameters (rhythm, timbre, signal-to-noise ratio) through neural network processing rather than relying on pre-recorded segments. This parameter-based generation ensures consistency with the original speaker's characteristics while maintaining editing efficiency
Solution Approach 2:
The neural network processes the original speech as input and generates speech features that feedback to ensure consistency with the original timbre and rhythm. The network learns from the original speech characteristics and applies them to the corrected segments, creating a feedback loop that maintains speaker consistency
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Embodiments of this application disclose a speech processing method and a related device, which may be applied to a scenario in which a user records a short video, a teacher records a teaching speech, or the like. The method includes: obtaining an original speech and a second text, where an original text corresponding to the original speech and a target text to which the second text belongs both include a first text; generating a second speech feature of a corrected text (that is, the second text) in the target text by referring to a first speech feature of a correct text (that is, the first text) in the original text; and then generating, based on the second speech feature, a target edited speech corresponding to the corrected text. Therefore, auditory perception of the target edited speech corresponding to a modified part is similar to auditory perception of a non-edited speech corresponding to the correct text, and user experience is improved.