Method for improving sound quality of synthetic speech based on harmonic enhancement model

Through adaptive harmonic detection and phase optimization algorithms, a harmonic enhancement model is built, which solves the problem of insufficient noise and harmonic processing of speech synthesis system under low-quality data, improves the sound quality and stability of speech synthesis, and is suitable for a variety of applications Scene.

CN119028370APending Publication Date: 2024-11-26GIANT MOBILE TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411247029.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-06
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

When processing low-quality data, existing speech synthesis systems are susceptible to noise, vibrato and electronic music, resulting in unnatural sound quality of synthetic speech. The traditional methods are not fine enough in harmonic processing to fully enhance the harmonic characteristics of speech.

Method used

Adaptive harmonic detection algorithm is used to extract the harmonic peak and amplitude in the speech signal, build a harmonic enhancement model, and eliminate vibrato and electronic music through the harmonic phase optimization algorithm, and fuse the enhanced harmonic characteristics to form a synthetic speech signal.

Benefits of technology

It significantly improves the clarity and nature of synthetic speech, reduces vibrato and electronic music, improves the stability and reliability of the model under low-quality data, and is suitable for a variety of speech synthesis applications.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119028370A_ABST
    Figure CN119028370A_ABST
Patent Text Reader

Abstract

The invention relates to a method for improving the tone quality of synthetic speech based on a harmonic enhancement model. The method comprises the following steps: S1, analyzing and extracting harmonic features; s2, a self-adaptive harmonic detection algorithm is adopted, the harmonic peak position and amplitude in the voice signals are accurately recognized, and effective capture of high-frequency harmonics is ensured; s3, constructing a harmonic wave enhancement model, and training the harmonic wave features detected by adopting an adaptive harmonic wave detection algorithm to obtain the harmonic wave enhancement model; and S4, fusing the harmonic characteristics enhanced by the harmonic enhancement model with the original voice signal to form a synthetic voice signal. According to the method, the overall quality of the synthesized speech is remarkably improved through fine processing and enhancement of the harmonic characteristics.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of speech synthesis, and in particular to a method for improving the sound quality of synthesized speech based on a harmonic enhancement model. Background Art

[0002] In speech synthesis technology, the sound quality of synthesized speech is closely related to the quality of training data. However, current speech synthesis systems face many challenges when processing low-quality data, mainly in the following aspects:

[0003] (1) Impact of data noise: Existing speech synthesis models are often constrained by low-quality training data, which may contain background noise, microphone distortion and other problems, resulting in the model being unable to accurately capture the original sound characteristics.

[0004] (2) Vibrato and electric sound problems: Vibrato and electric sound problems are common in synthesized speech and seriously affect the user experience. These problems are usually caused by the model's inability to capture pitch and timbre changes, especially when dealing with high-frequency harmonic components.

[0005] (3) Insufficient harmonic processing: Traditional methods are often not sophisticated enough in harmonic processing and cannot fully enhance the harmonic characteristics of speech, resulting in unnatural and unrealistic synthesized sound quality.

[0006] Therefore, it is necessary to provide a method for improving the sound quality of synthesized speech based on a harmonic enhancement model, which can significantly improve the overall quality of synthesized speech by fine-tuning and enhancing harmonic features. Summary of the invention

[0007] The purpose of the present invention is to provide a method for improving the sound quality of synthesized speech based on a harmonic enhancement model, so as to improve the overall quality of synthesized speech by finely processing and enhancing harmonic features.

[0008] In order to solve the problems existing in the prior art, the present invention provides a method for improving the sound quality of synthesized speech based on a harmonic enhancement model, comprising the following steps:

[0009] S1: Analyze and extract harmonic features;

[0010] S2: Adopts adaptive harmonic detection algorithm to accurately identify the harmonic peak position and amplitude in the voice signal, ensuring the effective capture of high-frequency harmonics;

[0011] S3: construct a harmonic enhancement model, train the harmonic features detected by the adaptive harmonic detection algorithm, and obtain the harmonic enhancement model;

[0012] S4: The harmonic features enhanced by the harmonic enhancement model are fused with the original speech signal to form a synthetic speech signal.

[0013] Optionally, in the method for improving the sound quality of synthesized speech based on the harmonic enhancement model, the manner of analyzing and extracting harmonic features is as follows: performing spectrum analysis on the input speech signal using Fourier transform to extract the basic frequency and harmonic components of the speech.

[0014] Optionally, in the method for improving the sound quality of synthesized speech based on the harmonic enhancement model, before analyzing and extracting the harmonic features, the following steps are further included:

[0015] Collect and preprocess training data, perform preliminary noise removal, and ensure data quality.

[0016] Optionally, the method for improving the sound quality of synthesized speech based on the harmonic enhancement model further includes the following steps:

[0017] Through the harmonic phase optimization algorithm, the phase information in the synthesized speech signal is adjusted to eliminate the vibrato and electric sound in the synthesized speech.

[0018] Optionally, the method for improving the sound quality of synthesized speech based on the harmonic enhancement model further includes the following steps:

[0019] According to the synthesized speech signal, its harmonic features are extracted, the newly extracted harmonic features are detected and trained, and the harmonic enhancement model is optimized.

[0020] Compared with the prior art, the present invention has the following advantages:

[0021] (1) Sound quality improvement: The harmonic characteristics are enhanced to improve the clarity and naturalness of the synthesized speech, making it closer to the human voice.

[0022] (2) Reducing vibrato and electric sound: Vibrato and electric sound in speech are eliminated through harmonic phase optimization, improving user experience.

[0023] (3) Improving model robustness: The performance of the harmonic enhancement model on low-quality datasets improves the stability and reliability of speech synthesis.

[0024] (4) Wide range of applications: Applicable to various speech synthesis applications, including intelligent voice assistants, voice navigation, voice education and other fields.

[0025] (5) Easy to integrate: The present invention can be seamlessly integrated with the existing speech synthesis system to enhance the sound quality performance of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 A flow chart of a method provided by an embodiment of the present invention;

[0027] Figure 2A flow chart of speech synthesis provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The specific implementation of the present invention will be described in more detail below in conjunction with the schematic diagram. The advantages and features of the present invention will become clearer based on the following description. It should be noted that the drawings are all in a very simplified form and are not in exact proportions, and are only used to facilitate and clearly assist in explaining the purpose of the embodiments of the present invention.

[0029] Hereinafter, if the method described herein includes a series of steps, the order in which the steps are presented herein is not necessarily the only order in which the steps may be performed, and some of the steps described may be omitted and / or some other steps not described herein may be added to the method.

[0030] In speech synthesis technology, the sound quality of synthesized speech is closely related to the quality of training data. However, current speech synthesis systems face many challenges when processing low-quality data, resulting in poor sound quality of synthesized speech.

[0031] In order to solve the problems existing in the prior art, the present invention provides a method for improving the sound quality of synthesized speech based on a harmonic enhancement model, such as Figure 1 As shown, the method comprises the following steps:

[0032] Collect and preprocess training data, perform preliminary noise removal, and ensure data quality.

[0033] S1: Analyze and extract harmonic features in the following way: Use Fourier transform to perform spectrum analysis on the input speech signal to extract the fundamental frequency (F0) and harmonic components of the speech.

[0034] S2: Adopts adaptive harmonic detection algorithm to accurately identify the harmonic peak position and amplitude in the voice signal, ensuring the effective capture of high-frequency harmonics;

[0035] According to the International Electrotechnical Commission IEC581 standard and my country's GB / T14277-93 national standard, the frequency band range is 30-150Hz for low-frequency harmonics; 150-500Hz for medium-frequency harmonics; 500-5KHz for medium- and high-frequency harmonics; and 5K-16KHz for high-frequency harmonics.

[0036] S3: Construct a harmonic enhancement model, train the harmonic features detected by the adaptive harmonic detection algorithm, and obtain a harmonic enhancement model, so as to enhance the harmonic components of the speech signal through harmonic compensation and filtering technology for low-quality speech data;

[0037] S4: The harmonic features enhanced by the harmonic enhancement model are fused with the original speech signal to form a synthetic speech signal.

[0038] Preferably, the method further comprises the following steps:

[0039] According to the synthesized speech signal, its harmonic features are extracted, the newly extracted harmonic features are detected and trained, and the parameters of the harmonic enhancement model are optimized to improve the generalization ability of the model.

[0040] Preferably, the method further comprises the following steps:

[0041] Through the harmonic phase optimization algorithm, the phase information in the synthesized speech signal is adjusted to eliminate the vibrato and electric sound in the synthesized speech. The phase recovery technology is used to improve the coherence and fluency of speech synthesis.

[0042] In one embodiment, reference Figure 2 , the process of synthesizing speech is as follows:

[0043] (1) Text Encoder:

[0044] Input: Accepts text input and other features such as speaker identity (spk_id) and sentiment embedding.

[0045] Output: Generate a feature representation that contains language content and speaker characteristics.

[0046] (2) Pitch Adaptor:

[0047] Input: Receives input from the text encoder.

[0048] Output: Generates a pitch-adapted mean of the latent variable, making the network focus on the pitch characteristics of the speech.

[0049] (3)Flow:

[0050] Input: Combination of text encoding features, enhanced pitch prediction and other acoustic parameters.

[0051] Function: Convert text encoding features into a latent space (z) that captures the necessary properties for speech synthesis.

[0052] Training: Interact with the posterior encoder features during training to learn rich representations.

[0053] (4) Posterior Encoder:

[0054] Input: Processed real audio features (g) and corresponding latent features (y).

[0055] Output: Generates a posterior distribution with mean (m_q) and log standard deviation (logs_q) to guide the learning of the latent space.

[0056] Purpose: To ensure that the latent space accurately models real speech characteristics during training.

[0057] (5) Harmonic Noise Superposition Function (hn-NSF):

[0058] Input: Latent features (z) are obtained from Flow and refined by harmonic enhancement.

[0059] Output: Refined audio features are generated, ready for decoding.

[0060] Function: Improve the naturalness and quality of synthesized speech by emphasizing harmonic components.

[0061] (6) Decoder:

[0062] Input: Receives enhanced latent features from hn-NSF.

[0063] Output: Generate the final waveform (wav), i.e. the synthesized speech.

[0064] Function: Combines learned harmonic structure to convert high-level representations into audible sounds.

[0065] In summary, compared with the prior art, the present invention has the following advantages:

[0066] (1) Sound quality improvement: The harmonic characteristics are enhanced to improve the clarity and naturalness of the synthesized speech, making it closer to the human voice.

[0067] (2) Reducing vibrato and electric sound: Vibrato and electric sound in speech are eliminated through harmonic phase optimization, improving user experience.

[0068] (3) Improving model robustness: The performance of the harmonic enhancement model on low-quality datasets improves the stability and reliability of speech synthesis.

[0069] (4) Wide range of applications: Applicable to various speech synthesis applications, including intelligent voice assistants, voice navigation, voice education and other fields.

[0070] (5) Easy to integrate: The present invention can be seamlessly integrated with the existing speech synthesis system to enhance the sound quality performance of the system.

[0071] The above is only a preferred embodiment of the present invention and does not limit the present invention in any way. Any technician in the relevant technical field, without departing from the scope of the technical solution of the present invention, makes any form of equivalent replacement or modification to the technical solution and technical content disclosed in the present invention, which does not depart from the content of the technical solution of the present invention and still falls within the protection scope of the present invention.

Claims

1. A method for improving the sound quality of synthesized speech based on a harmonic enhancement model, characterized in that: The following steps are involved: S1: Analyze and extract harmonic features; S2: Adopts adaptive harmonic detection algorithm to accurately identify the harmonic peak position and amplitude in the voice signal, ensuring the effective capture of high-frequency harmonics; S3: construct a harmonic enhancement model, train the harmonic features detected by the adaptive harmonic detection algorithm, and obtain the harmonic enhancement model; S4: The harmonic features enhanced by the harmonic enhancement model are fused with the original speech signal to form a synthetic speech signal.

2. The method for improving the sound quality of synthesized speech based on the harmonic enhancement model as claimed in claim 1, characterized in that: The method of analyzing and extracting harmonic features is as follows: Spectral analysis of the input speech signal is performed using Fourier transform to extract the basic frequency and harmonic components of the speech.

3. The method for improving the sound quality of synthesized speech based on the harmonic enhancement model as claimed in claim 1, characterized in that: Before analyzing and extracting harmonic features, the following steps are also included: Collect and preprocess training data, perform preliminary noise removal, and ensure data quality.

4. The method for improving the sound quality of synthesized speech based on the harmonic enhancement model as claimed in claim 1, characterized in that: The following steps are also included: Through the harmonic phase optimization algorithm, the phase information in the synthesized speech signal is adjusted to eliminate the vibrato and electric sound in the synthesized speech.

5. The method for improving the sound quality of synthesized speech based on the harmonic enhancement model as claimed in claim 1, characterized in that: The following steps are also included: According to the synthesized speech signal, its harmonic features are extracted, the newly extracted harmonic features are detected and trained, and the harmonic enhancement model is optimized.