Method for constructing multi-dialect fusion TTS based on mandarin ability

Abstract

The invention relates to a method for constructing a multi-dialect fusion TTS based on mandarin ability. The method comprises the following steps: S1, adopting phonemes to mark dialect voices; s2, presetting a target dialect extension module based on the voice marking scheme; s3, inputting the text into a target dialect text regularization module to obtain a text with all readable characters, calling a front-end module of the mandarin TTS to convert the text with all readable characters into front-end information, and inputting the front-end information into a target dialect extension module; s4, the target dialect extension module corrects the Chinese character mandarin pronunciation in the front-end information into the target dialect pronunciation; s5, integrating the mandarin and various target dialect voice marking schemes to establish a shared rear-end phoneme space, and creating different language numbers for the mandarin and various dialects; and S6, based on the audio data of mandarin and various dialects, phoneme marks and language numbers, training a TTS basic model with various dialect capabilities. According to the invention, the cost of developing the dialect TTS can be reduced.

Description

Technical Field

[0001] The present invention relates to the field of speech fusion technology, and in particular to a method for constructing a multi-dialect fusion TTS based on Mandarin capabilities. Background Art

[0002] Most Chinese dialects have varying tones, finals, and initials that are not found in Mandarin. Developing a dialect TTS (Text-to-Speech) system using existing technologies requires generating sufficient audio data in that dialect to build the backend. However, high-quality dialect audio is often difficult to collect, and existing technologies require a high investment to synthesize natural dialect speech in a stable and controllable manner.

[0003] Therefore, it is necessary to provide a method for building multi-dialect fusion TTS based on Mandarin capabilities, which can reduce the cost of developing dialect TTS by expanding the backend phoneme space and reusing multi-dialect audio corpora based on Mandarin capabilities. Summary of the Invention

[0004] The purpose of the present invention is to provide a method for constructing a multi-dialect fusion TTS based on Mandarin capabilities, which reduces the cost of developing dialect TTS by expanding the backend phoneme space and reusing multi-dialect audio corpora based on Mandarin capabilities.

[0005] In order to solve the problems existing in the prior art, the present invention provides a method for constructing a multi-dialect fusion TTS based on Mandarin capabilities, comprising the following steps:

[0006] S1: Using phonemes to mark dialect speech to determine the speech marking schemes of various target dialects;

[0007] S2: preset target dialect extension module based on the phonetic tagging scheme;

[0008] S3: Inputting the text into the target dialect text regularization module to obtain a text with all characters readable, calling the Mandarin TTS front-end module to convert the text with all characters readable into front-end information, and inputting the front-end information into the target dialect expansion module;

[0009] S4: The target dialect expansion module corrects the Mandarin pronunciation of the Chinese characters in the front-end information into the target dialect pronunciation to achieve the generation of the target dialect front-end data;

[0010] S5: Integrate the speech tagging schemes of Mandarin and various target dialects to establish a common backend phoneme space, and create different language numbers for Mandarin and various dialects;

[0011] S6: Train a TTS base model with multi-dialect capabilities based on audio data, phoneme tags, and language numbers of Mandarin and various dialects.

[0012] Optionally, in the method of constructing multi-dialect fusion TTS based on Mandarin language capabilities, the target dialect extension module includes a dialect character-phonetic replacement table, a word-phonetic replacement table, a dialect tone change rule, a dialect polyphone disambiguation model, a dialect word tone generation model, a dialect tone conversion table, and a dialect syllable analysis table.

[0013] Optionally, in the method for constructing multi-dialect fusion TTS based on Mandarin language capabilities, the TTS basic model is updated in a timely manner according to the target dialect.

[0014] Optionally, in the method for constructing multi-dialect fusion TTS based on Mandarin language capabilities, updating the TTS basic model includes the following steps:

[0015] If all the phonemes in the target dialect already exist in the public backend phoneme space, speech synthesis in the target dialect is achieved based on TTS basic model training;

[0016] If there are new phonemes outside the public backend phoneme space in the target dialect, the public backend phoneme space is updated, and more data containing the new phonemes is collected. The updated TTS basic model is retrained together with the data used to train the TTS basic model previously.

[0017] Compared with the prior art, the present invention has the following advantages:

[0018] The present invention is based on Mandarin capabilities, reduces the cost of developing dialect TTS by expanding the backend phoneme space and reusing multi-dialect audio corpora. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A flow chart of a method provided by an embodiment of the present invention;

[0020] Figure 2 This is a flowchart of updating the TTS basic model provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0021] The following is a more detailed description of the specific embodiments of the present invention with reference to schematic diagrams. The advantages and features of the present invention will become more apparent from the following description. It should be noted that the drawings are greatly simplified and not to exact scale, and are only used for the purpose of conveniently and clearly illustrating the embodiments of the present invention.

[0022] Hereinafter, if the method described herein includes a series of steps, the order in which the steps are presented herein is not necessarily the only order in which the steps may be performed, and some of the steps described may be omitted and / or some other steps not described herein may be added to the method.

[0023] Most Chinese dialects have varying tones, finals, and initials that are not found in Mandarin. Developing a dialect TTS (Text-to-Speech) system using existing technologies requires generating sufficient audio data in that dialect to build the backend. However, high-quality dialect audio is often difficult to collect, and existing technologies require a high investment to synthesize natural dialect speech in a stable and controllable manner.

[0024] In order to solve the problems existing in the prior art, the present invention provides a method for constructing a multi-dialect fusion TTS based on Mandarin capabilities, such as Figure 1 As shown, the following steps are included:

[0025] S1: Using phonemes to mark dialect speech to determine the speech marking schemes of various target dialects;

[0026] Specifically, we ensure that for each dialect, every phoneme (initial or final) can be labeled using either existing Mandarin phoneme symbols or newly established phoneme symbols, without disrupting the topological structure of the dialect's speech system. Based on this, we determine the speech labeling schemes for each target dialect, and based on these labeling schemes, we generate audio corpora and polyphonetic word disambiguation corpora for the target dialects as training data.

[0027] S2: Preset the target dialect extension module based on the phonetic tagging scheme. The target dialect extension module includes a dialect word-phonetic substitution table, a word-phonetic substitution table, a dialect tone change rule, a dialect polyphonetic word disambiguation model, a dialect word-tone generation model, a dialect tone conversion table, and a dialect syllable analysis table.

[0028] S3: Inputting the text into the target dialect text normalization module to obtain a text with all characters readable, calling the Mandarin TTS front-end module to convert the text with all characters readable into front-end information, and inputting the front-end information into the target dialect extension module;

[0029] S4: The target dialect expansion module corrects the Mandarin pronunciation of the Chinese characters in the front-end information into the target dialect pronunciation to achieve the generation of the target dialect front-end data;

[0030] S5: Integrate the speech tagging schemes of Mandarin and various target dialects to establish a common backend phoneme space, and create different language numbers for Mandarin and various dialects;

[0031] S6: A TTS basic model with multi-dialect capabilities is trained based on audio data, phoneme tags, and language numbers of Mandarin and various dialects. This TTS basic model uses a shared backend phoneme space to reuse the precise labeled data of Mandarin and different dialects as much as possible, ensuring that the phonemes in the phoneme space achieve Mandarin-level speech generation quality.

[0032] After the expansion of the above front-end and back-end modules, a model for synthesizing human voice in the target dialect is obtained.

[0033] Preferably, Figure 2 As shown, updating the TTS basic model in a timely manner according to the target dialect includes the following steps:

[0034] If all the phonemes in the target dialect already exist in the public backend phoneme space, speech synthesis in the target dialect is achieved based on TTS basic model training;

[0035] If there are new phonemes outside the public backend phoneme space in the target dialect, the public backend phoneme space is updated, and more data containing the new phonemes is collected. The updated TTS basic model is retrained together with the data used to train the TTS basic model previously.

[0036] In summary, compared with the prior art, the present invention has the following advantages:

[0037] The present invention is based on Mandarin capabilities, reduces the cost of developing dialect TTS by expanding the backend phoneme space and reusing multi-dialect audio corpora.

[0038] The above description is merely a preferred embodiment of the present invention and does not limit the present invention in any way. Any person skilled in the art who, without departing from the scope of the present invention, makes any equivalent substitution, modification, or other changes to the technical solution and technical content disclosed in the present invention shall be deemed to be within the scope of the present invention and still fall within the scope of protection of the present invention.

Claims

1. A method for constructing multi-dialect fusion TTS based on Mandarin ability, characterized in that: The following steps are involved: S1: Using phonemes to mark dialect speech to determine the speech marking schemes of various target dialects; S2: preset target dialect extension module based on speech markup scheme; S3: inputting the text into the target dialect text regularization module to obtain a text with all characters readable, calling the front-end module of Mandarin TTS to convert the text with all characters readable into front-end information, and inputting the front-end information into the target dialect expansion module; S4: the target dialect expansion module corrects the Mandarin pronunciation of the Chinese characters in the front-end information into the target dialect pronunciation, so as to realize the generation of the target dialect front-end data; S5: A common backend phoneme space is established by integrating the speech tagging schemes of Mandarin and various target dialects, and different language numbers are created for Mandarin and various dialects; S6: Train a TTS base model with multi-dialect capabilities based on audio data, phoneme tags, and language numbers of Mandarin and various dialects.

2. The method for constructing multi-dialect fusion TTS based on Mandarin ability as claimed in claim 1, characterized in that: The target dialect extension module includes a dialect character-phonetic substitution table, a word-phonetic substitution table, a dialect tone change rule, a dialect polyphone disambiguation model, a dialect word tone generation model, a dialect tone conversion table, and a dialect syllable analysis table.

3. The method for constructing multi-dialect fusion TTS based on Mandarin ability as claimed in claim 1, characterized in that: Update the TTS basic model in a timely manner according to the target dialect.

4. The method for constructing multi-dialect fusion TTS based on Mandarin ability as claimed in claim 3, characterized in that: Updating the TTS basic model includes the following steps: If all the phonemes in the target dialect already exist in the public backend phoneme space, speech synthesis of the target dialect is achieved based on TTS basic model training; If there are new phonemes outside the common backend phoneme space in the target dialect, the common backend phoneme space is updated, and more data containing the new phonemes are collected and retrained together with the data used to train the TTS basic model before to obtain an updated TTS basic model.

Images