A multi-style personalized Tibetan speech synthesis model for low-resource conditions

Abstract

The present invention discloses a multi-style personalized Tibetan speech synthesis model for low-resource conditions, including a speaker style encoder and a sequentially connected grapheme-to-phoneme conversion module, a text encoder, a variance adapter, a mel spectrum decoder and a HifiGAN vocoder; the synthesis model is trained using a "pre-training + meta-learning" model algorithm; the model parameters that can be learned by the deep learning model are converted to i Divided into parameters related to rhythm i p , parameters related to the speaker i s and other remaining parameters #imgabs0#, i.e. #imgabs1# i p and i s The model includes learnable model parameters for the speaker style encoder, and #imgabs2# includes learnable model parameters for the text encoder, mel-spectrogram decoder, and variance adapter. This synthetic model can be applied to intelligent bilingual teaching products, as well as Tibetan spoken dialogue, multimedia information processing, and other human-computer interaction fields.

Description

Technical Field

[0001] The present invention belongs to the technical field of speech synthesis and relates to a Tibetan speech synthesis model, and in particular to a multi-style personalized Tibetan speech synthesis model under low-resource conditions. Background Art

[0002] Text-to-speech (TTS), also known as text-to-speech synthesis, has long been a research hotspot in intelligent speech technology. my country is a multi-ethnic country with a rich diversity of minority languages ​​and local dialects, in addition to Mandarin Chinese. While speech synthesis technology for Mandarin Chinese is relatively mature, overall research in minority languages ​​remains limited. This is particularly due to a weak linguistic research foundation and a lack of standard speech corpora. Further improvements are needed in terms of the naturalness and personalization of synthesized speech. Tibetan is widely spoken by the Tibetan people in my country. However, due to geographical, cultural, and language limitations, economic, cultural, and educational development in Tibetan areas is extremely uneven. Using speech synthesis technology to achieve Tibetan speech synthesis in Tibetan areas will promote the development of Tibetan information processing technology and have important implications for promoting ethnic communication and unity and preserving minority cultures.

[0003] Currently, computer speech synthesis primarily utilizes statistical parameter-based speech synthesis methods and end-to-end speech synthesis methods. Statistical parameter-based speech synthesis methods include those based on the Hidden Markov Model (HMM) and those based on deep learning. HMM-based statistical parameter speech synthesis methods reconstruct overly smooth signal spectral envelopes, resulting in dull and lacking rhythmic synthesized speech. Compared to HMM-based statistical parameter speech synthesis methods, deep learning-based statistical parameter speech synthesis methods utilize deep learning techniques to achieve a nonlinear mapping between linguistic contextual features and acoustic features. Consequently, deep learning-based statistical parameter speech synthesis methods achieve superior speech quality compared to HMM-based methods. Statistical parameter-based speech synthesis systems typically consist of a text analysis front-end, a duration prediction model, an acoustic parameter prediction model, and a complex signal processing-based vocoder. The design of these components often requires specialized domain knowledge, and each module is trained separately, which can easily lead to cumulative errors.

[0004] End-to-end speech synthesis has become a major research hotspot in speech synthesis. End-to-end speech synthesis methods can be broadly categorized into autoregressive end-to-end speech synthesis, represented by Tacotron2 and DeepVoice3, and non-autoregressive end-to-end speech synthesis, represented by Paranet and Fastspeech2. Autoregressive end-to-end speech synthesis methods suffer from slow synthesis inference speed and low robustness (prone to word skipping and missing), while non-autoregressive models such as Paranet and Fastspeech2 achieve both fast inference and high robustness. Advances in end-to-end speech synthesis technology have enabled the synthesis of high-quality, speaker-specific, neutral speech for mainstream languages ​​(such as English, Mandarin Chinese, French, and German). However, multi-style, personalized speech synthesis still struggles to meet practical needs. Existing technologies for multi-style, personalized speech synthesis in mainstream languages ​​primarily employ speaker-adaptive training and speaker-style encoding. The speaker adaptation training method first pre-trains an acoustic model using large-scale multi-speaker corpora. It then fine-tunes all or part of the model parameters of the pre-trained acoustic model using a small amount of target speaker corpora. Speaker style encoding uses a speaker style encoder to extract a style embedding vector representing the speech style and a speaker embedding vector representing the speaker identity from speech samples. These style embeddings and speaker embeddings control the encoding and decoding process of the end-to-end speech synthesis system, thereby achieving multi-style and personalized speech synthesis.

[0005] There are approximately 6,000 languages ​​spoken worldwide. Currently, popular commercial speech synthesis services only support a few dozen mainstream languages, such as English, Mandarin Chinese, French, and German. Compared to mainstream languages, the vast majority of other languages ​​are considered low-resource languages. This has two implications: First, some are endangered. Second, despite having relatively large populations, some languages ​​face challenges such as a weak linguistic research foundation and limited access to standard speech corpora due to economic, regional, and cultural disparities. Furthermore, in information processing research on mainstream languages, the difficulty in obtaining training data for certain specialized tasks (such as multi-style speech synthesis in mainstream languages) also aligns with the low-resource scenario. In speech synthesis for low-resource languages, given the weak linguistic research foundation and limited access to standard speech corpora, a "pre-training + fine-tuning" approach is employed. This approach leverages the similarities between language pronunciations, first pre-training an acoustic model using a large corpus, and then fine-tuning all or part of the pre-trained model's parameters using a small amount of target language data to achieve speech synthesis for low-resource languages. Tibetan speech synthesis faces not only the common challenges associated with speech synthesis research for low-resource languages, but also two additional challenges: 1) Although Tibetan is a phonetic language, it possesses a unique two-dimensional writing structure. Therefore, a specialized text analysis front-end is required. 2) Tibet is vast and sparsely populated. While the written language remains standardized, the pronunciation of spoken Tibetan varies significantly across regions. Therefore, compared to other languages, multi-style, personalized speech synthesis in Tibetan faces a more specialized text analysis front-end and the difficulty in obtaining standard multi-style corpora. Currently, no multi-style, personalized Tibetan speech synthesis model has been developed for low-resource environments. Summary of the Invention

[0006] The purpose of the present invention is to provide a multi-style personalized Tibetan speech synthesis model under low-resource conditions.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is: a multi-style personalized Tibetan speech synthesis model for low-resource conditions, characterized by comprising a speaker style encoder and a grapheme-to-phonetic conversion module, a text encoder, a variance adapter, a mel-spectrogram decoder, and a HifiGAN vocoder connected in sequence;

[0008] The speaker style encoder includes a speaker vector affine layer, a prosody vector extractor, a gradient flipping layer, and a speaker classifier, which are connected in sequence. The prosody vector extractor is also connected to the prosody vector affine layer. The prosody vector extractor consists of a fully connected layer A, a one-dimensional convolutional layer A, a multi-head attention mechanism layer A, a fully connected layer B, and a temporal information pooling layer, which are arranged in sequence. The temporal information pooling layer is connected to the gradient flipping layer and the prosody vector affine layer respectively.

[0009] The text encoder consists of a multi-head attention mechanism layer B, a speaker style adaptive normalization layer A, a one-dimensional convolution layer B, and a speaker style adaptive normalization layer B, which are arranged in this order; the multi-head attention mechanism layer B is connected to the word-to-sound conversion module; the speaker style adaptive normalization layer B is connected to the variance adapter;

[0010] The mel spectrum decoder consists of a multi-head attention mechanism layer C, a speaker style adaptive normalization layer C, a one-dimensional convolution layer C, a speaker style adaptive normalization layer D, and a fully connected layer C. The fully connected layer C is connected to the HifiGAN vocoder; the multi-head attention mechanism layer C is connected to the variance adapter;

[0011] The speaker style encoder is connected to the text encoder and the mel spectrum decoder respectively;

[0012] The synthetic model is trained using the "pre-training + meta-learning" model algorithm; the model parameters that can be learned by the deep learning model are i Divided into parameters related to rhythm i p , parameters related to the speaker i s and other remaining parameters Composition, that is ; i p and i s including learnable model parameters of the speaker style encoder, Includes learnable model parameters for text encoder, mel-spectrogram decoder, and variance adapter.

[0013] The proposed multi-style personalized Tibetan speech synthesis model aims to achieve multi-style personalized Tibetan speech synthesis under low-resource conditions. It primarily encompasses two aspects: 1) It examines the representation mechanism of the target speaker's characteristic sound space within an end-to-end speech synthesis framework and proposes a multi-style personalized Tibetan speech synthesis model for low-resource conditions. This model primarily comprises a text encoder, a mel-spectrogram decoder, a variance adapter, a speaker style encoder, and a grapheme-to-phonetic conversion module to achieve multi-style personalized Tibetan speech synthesis under low-resource conditions. 2) It employs a "pre-training + meta-learning" approach and proposes a meta-learning model training algorithm based on pre-training to complete the training of the multi-style personalized Tibetan speech synthesis model for low-resource conditions. First, a multi-speaker neutral Mandarin acoustic model is pre-trained using a large-scale multi-speaker Mandarin corpus. A two-stage model-agnostic meta-learning (MAML) training is then performed using a smaller multi-speaker multi-style Mandarin corpus and a multi-speaker neutral Tibetan corpus. Finally, using a small amount of Tibetan target speaker corpus, adaptive training is performed in just a few steps to achieve multi-style personalized Tibetan speech synthesis under low-resource conditions. Furthermore, the training process decouples the speaker prosodic and timbre feature spaces within the end-to-end speech synthesis codec framework. This synthesis model has promising applications in intelligent bilingual teaching products, as well as in Tibetan oral dialogue, multimedia information processing, and other fields of human-computer interaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a speech synthesis model diagram of the present invention.

[0015] Figure 2 This is a flowchart of word-to-sound conversion in the speech synthesis model of the present invention.

[0016] Figure 3 Schematic diagram of a variance adapter in the speech synthesis model of the present invention.

[0017] Figure 4 This is a schematic diagram of the model training process based on "pre-training + meta-learning". DETAILED DESCRIPTION

[0018] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0019] The main limitations of existing multi-style, personalized speech synthesis methods for mainstream languages ​​are: 1) End-to-end speech synthesis methods require large-scale training datasets, but large-scale, multi-style emotional corpora are difficult to obtain. 2) Although personalized speech synthesis has been achieved, the model generalization ability is poor, that is, the adaptation performance to unseen speakers is poor, especially when the reference speech length is short. For speech synthesis in low-resource languages, end-to-end speech synthesis requires more training corpora than speech synthesis based on statistical parameters, and constructing large-scale corpora for speech synthesis in low-resource languages ​​(such as Tibetan) is complex and difficult. Although the "pre-training + fine-tuning" method can achieve Tibetan speech synthesis under low-resource conditions, it still suffers from shortcomings such as low synthesized speech similarity and weak expressiveness.

[0020] In order to overcome the problems existing in the above-mentioned prior art, the present invention provides a Figure 1 The multi-style personalized Tibetan speech synthesis model under low-resource conditions shown in the figure includes a speaker style encoder and a sequentially connected grapheme-to-phonetic conversion module, a text encoder, a variance adapter, a mel-spectrogram decoder, and a HifiGAN vocoder;

[0021] The speaker style encoder includes a speaker vector affine layer and a prosody vector extractor, a gradient flipping layer, and a speaker classifier connected in sequence. The prosody vector extractor is also connected to the prosody vector affine layer; the prosody vector extractor is composed of a fully connected layer A, a one-dimensional convolutional layer A, a multi-head attention mechanism layer A, a fully connected layer B, and a temporal information pooling layer arranged in sequence; the temporal information pooling layer is connected to the gradient flipping layer and the prosody vector affine layer respectively.

[0022] The text encoder consists of a multi-head attention mechanism layer B, a speaker-adaptive normalization layer A, a one-dimensional convolution layer B, and a speaker-adaptive normalization layer B, which are arranged in this order. The multi-head attention mechanism layer B is connected to the grapheme-to-speech conversion module; the speaker-adaptive normalization layer B is connected to the variance adapter.

[0023] The mel-spectrogram decoder consists of a multi-head attention mechanism layer C, a speaker-style adaptive normalization layer C, a one-dimensional convolutional layer C, a speaker-style adaptive normalization layer D, and a fully connected layer C. The fully connected layer C is connected to the HifiGAN vocoder; the multi-head attention mechanism layer C is connected to the variance adapter.

[0024] The speaker style encoder is connected to the text encoder and the mel spectrum decoder respectively;

[0025] Although Tibetan is a phonetic language, it has a unique two-dimensional writing structure. The grapheme-to-phonetic conversion module converts Tibetan text into computer-readable phonetic symbols.

[0026] The synthesis model of the present invention adopts the existing Tibetan computer readable phonetic symbol SAMPA-T scheme, according to Figure 2 The process shown is to perform a grapheme-to-phonetic conversion to obtain the initials and finals of a Tibetan character syllable. First, a single Tibetan character is obtained from a Tibetan syllable by cutting the separating sound "་"; then, the obtained single Tibetan character is located by the base character, and the Tibetan character is decomposed into the pre-added character, the base character, the post-added character, and the post-added character; then, the base character is decomposed into the pre-added character, the base character, the post-added character, and the vowel by looking up the table; then, the pre-added character, the pre-added character, the base character, the post-added character, and the vowel are converted into the initial consonant combination by looking up the initial consonant SAMPA-T dictionary, and the vowel, the post-added character, and the post-added character are converted into the final consonant combination by using the final consonant SAMPA-T dictionary. After that, the initial consonant combination and the final consonant combination realize the transcription of the Tibetan character to the initial and final consonant SAMPA-T.

[0027] The combination rules of Tibetan initials and finals are as follows:

[0028] Tibetan initial consonant = pre-added character + base character without vowel (pre-added character + base character + post-added character)

[0029] Tibetan vowel = vowel + suffix + suffix

[0030] Table 1 shows the main hyperparameter configurations of the model for multi-style personalized Tibetan speech synthesis under low-resource conditions.

[0031] Table 1. Main hyperparameter configurations of the model for multi-style personalized Tibetan speech synthesis under low-resource conditions

[0032]

[0033] The prosody vector of the input reference speech is extracted frame by frame through the prosody vector extractor (the prosody vector can be extracted from the reference speech Mel spectrogram in the speaker style encoder in the figure. The reference speech Mel spectrogram is actually another representation of the reference speech). The prosody vector is transformed by the prosody vector affine layer to obtain the parameters representing the prosodic features of the speech. ;

[0034] The speaker ID encoded in one-hot encoding is converted into a speaker vector through a trainable speaker look-up table (LUT). The speaker vector is transformed through the speaker vector affine layer to obtain the parameters representing the speaker's timbre characteristics. ;

[0035] Parameters representing the prosodic features of speech and parameters representing the speaker's timbre characteristics Synthesized parameter sets Used to modulate the hidden feature representations in the text encoder and mel-spectrogram decoder to control the timbre and prosodic characteristics of the synthesized speech speaker.

[0036] At the same time, by adopting the principle of domain adversarial training, a gradient reversal layer (GRL) and a speaker classifier are added to the speaker style encoder to fully separate the speaker information carried by the prosody vector.

[0037] The text encoder converts the phoneme embedding sequence output by the grapheme-to-phoneme conversion module into a phoneme hidden sequence. A variance adapter then adds different variance information (such as duration, fundamental frequency, and energy) to the hidden sequence. Finally, the mel-spectrogram decoder converts the adjusted hidden sequence into a mel-spectrogram sequence in parallel. The text encoder and mel-spectrogram decoder are constructed by stacking an improved N-layer feed-forward transformer (FFT) building block. This improved FFT structure sequentially consists of a multi-head attention mechanism layer, a speaker-adaptive normalization layer, a one-dimensional convolutional layer, and a speaker-adaptive normalization layer. This improved FFT structure differs from the prior art FFT in Fastspeech2 in that the synthesis method of the present invention replaces the residual normalization layer in the Fastspeech2 FFT with a speaker-adaptive normalization layer. Generally, in end-to-end speech synthesis, the speaker vector (or speaker prosody vector) is simply concatenated with the encoder output vector (or decoder input vector), which does not fully and effectively represent the style characteristics of the relevant speaker. In the present invention, the speaker style adaptive normalization layer uses the speaker vector and prosody vector output by the speaker style encoder to perform style modulation on the feature vectors of the text encoder and the mel spectrum decoder, which more reasonably and effectively represents the style characteristics of the relevant speaker. Specifically, the speaker style adaptive normalization layer uses the parameter set representing the speaker's timbre characteristics Parameter set related to representing prosodic features The hidden features of the modulated text encoder and Mel spectrum decoder are shown in Equations (1) to (4).

[0038]

[0039] Among them, the speaker style adaptive normalization layer has a total of M layer, y i Indicates the i Layer Speaker Style Adaptation Normalization Layer Input Hidden State The normalized vector of N is the vector dimension; Indicates the i The output hidden state of the speaker style adaptation normalization layer, y i Normalize according to formula (2) to formula (4); m i and s i Respectively The mean and variance of .

[0040] The variance adapter adds the variance information of various speech signals (such as the duration, fundamental frequency, and energy of the initial and final vowels) to the hidden sequence output by the text encoder to improve the expressiveness of the synthesized speech. The variance adapter model of Fastspeech2 is used in the synthesis model of the present invention. Figure 2 As shown in Figure 1, the system primarily consists of a duration predictor, a fundamental frequency predictor, and an energy predictor. During training, the actual values ​​of the duration of the initials and finals, the fundamental frequency of the sentence, and the energy of the speech frame are extracted from the training dataset to train the duration predictor, fundamental frequency predictor, and energy predictor, respectively. During the inference phase, the predicted outputs of the duration predictor, fundamental frequency predictor, and energy predictor are used to synthesize the target speech.

[0041] The model structure of the duration predictor, fundamental frequency predictor and energy predictor is the same, which consists of a two-layer 1D convolutional network, a normalization layer, a dropout layer and a linear projection layer arranged in sequence.

[0042] Completed using the "pre-training + meta-learning" model training algorithm Figure 1 The figure shows the training of a multi-style personalized Tibetan speech synthesis model under low-resource conditions.

[0043] Meta-learning, or “learning to learn”, is a new machine learning method that has been booming in recent years. The meta-learning method based on MAML aims to find the sensitive initial points of model parameters through cross-task meta-training, so that a high model generalization ability can be achieved on new tasks with only a few steps of gradient updates. The synthetic model of this invention converts the model parameters that can be learned by the deep learning model into i Divided into rhythm-related parameters i p , speaker-related parameters i s and other remaining parameters It consists of three parts, namely . i p and i s including learnable model parameters of the speaker style encoder, Includes learnable model parameters for text encoder, mel-spectrogram decoder, and variance adapter.

[0044] Completed using a model training algorithm based on "pre-training + meta-learning" Figure 1 The training of the speech synthesis model of the present invention is shown in FIG. Figure 4As shown in the figure, it mainly includes four processes: pre-training stage, meta-learning stage 1, meta-learning stage 2 and adaptation stage.

[0045] In the pre-training phase, a multi-speaker Mandarin acoustic model is pre-trained using large-scale multi-speaker Mandarin corpus. i 0 represents the initialized acoustic model parameters, i M1 Represents the set of Mandarin acoustic model parameters after pre-training. At this stage, large-scale multi-speaker Mandarin corpora are readily available, including the Biaobei Open Source Mandarin Corpus and the Shilbeck Mandarin Corpus.

[0046] In the first meta-learning phase, based on the pre-trained multi-speaker Mandarin acoustic model, a small-scale multi-style multi-speaker Mandarin corpus is used to train a multi-style personalized Mandarin acoustic model using a meta-learning algorithm. i M2 represents the parameter set of the multi-style personalized Mandarin acoustic model obtained through meta-learning stage 1. At this stage, the difficulty of constructing large-scale multi-style, multi-speaker Mandarin corpora was overcome by the meta-learning algorithm. Currently, available small-scale multi-style, multi-speaker Mandarin corpora include Zhejiang University's EMOIVE and the National University of Singapore's ESD.

[0047] In the second meta-learning phase, based on the multi-style personalized Mandarin acoustic model, a small-scale Tibetan corpus is used to train a multi-style personalized Tibetan acoustic model using a meta-learning algorithm. i T Represents the parameter set of the multi-style personalized Tibetan acoustic model obtained through meta-learning stage 2.

[0048] In the speech synthesis model of the present invention, it is assumed that the <text, speech> data pairs of all corpora are represented as ( X , Y ), in the model training phase, to minimize the loss Target

[0049]

[0050] (5) In the formula, X Represents a text sequence, Y represents the real speech Mel spectrum, Represents the predicted speech Mel spectrum value, represents the L1 norm; represents the speaker classifier loss in the speaker style encoder. In the pre-training phase, all learnable model parameters are transformed by Updated to .

[0051] In the meta-learning stage 1, a small-scale multi-style and multi-speaker Mandarin corpus is labeled as a meta-dataset D meta-m , D meta-m Divide into meta-task training sets and meta-task test set , freeze parameter set i M1 Parameters in , only update speaker related parameters and rhythm-related parameters , so that the learnable model parameters are Updated to .for K -shot meta-learning tasks T i The speech synthesis model of the present invention collects a speaker's K <text, voice> data pairs to form , collect another K samples to form For a gradient update, update the parameters according to equations (6) to (7):

[0052]

[0053] in, α 1 is the learning rate of 1-task training in the meta-learning stage, Represents the speaker-related parameters of the multi-style personalized Mandarin acoustic model, Represents the prosody-related parameters of the multi-style personalized Mandarin acoustic model. Represents the speaker-related parameters of the multi-style personalized Mandarin acoustic model obtained in a task, Indicates that in a task T i The prosody-related parameters of the multi-style personalized Mandarin acoustic model obtained in . Indicates the loss of a task Find the gradient of the speaker-related parameters, Indicates a task T i loss Find the gradient of rhythm-related parameters.

[0054] In the meta-learning process, the parameters of the meta-learner are finally updated by calculating the loss gradient of the test task set, as shown in Equations (8) to (9):

[0055]

[0056] in,β 1 is the learning rate for 1-meta-task testing in the meta-learning stage.

[0057] In the meta-learning stage 2, the small-scale Tibetan corpus is labeled as a meta-dataset D meta-T , D meta-T Divide into meta-task training sets and meta-task test set , freeze parameter set i M2 Speaker style related parameters in , only update speaker related parameters and other parameters , so that the learnable model parameters are Updated to .for K -shot meta-learning tasks T i , the synthesis method of the present invention collects a speaker's K <text, voice> data pairs to form , collect another K samples to form For a gradient update, update the parameters according to (10) to (11):

[0058]

[0059] in, α 2 is the learning rate of 2-task training in the meta-learning stage, Represents the speaker-related parameters of the multi-style personalized Tibetan acoustic model, Represents other parameters of the multi-style personalized Tibetan acoustic model. Indicates that in a task T i Speaker-related parameters of the multi-style personalized Tibetan acoustic model obtained in Indicates that in a task T i Other parameters of the multi-style personalized Tibetan acoustic model obtained in . Indicates a task T i loss Find the gradients of other parameters. In the meta-learning process, the parameters of the meta-learner are finally updated by calculating the loss gradients of a batch of test task sets, as shown in Equations (12) to (13):

[0060]

[0061] in,β 2 is the learning rate for the 2-meta-task test in the meta-learning stage.

[0062] During the adaptation phase, the parameter set is frozen i T Prosody-related parameters in and other model parameters ,use K samples of the target speaker, and only update the speaker-related parameters , so that the learnable model parameters are Updated to . Updated by the loss gradient of the target speaker sample as shown in formula (14) ;

[0063]

[0064] in, α 3 adaptive phase meta-test learning rate, represents the speaker-related parameters of the multi-style personalized Tibetan target speaker acoustic model, Express K The loss of samples from the target speaker Find the gradient of speaker-dependent parameters.

[0065] The above training process is as follows:

[0066]

[0067]

[0068]

[0069]

[0070] .

Claims

1. A multi-style personalized Tibetan speech synthesis model for low-resource conditions, characterized by: It includes a speaker style encoder and a sequentially connected grapheme-to-speech conversion module, a text encoder, a variance adapter, a mel spectrum decoder, and a HifiGAN vocoder; The speaker style encoder includes a speaker vector affine layer, a prosody vector extractor, a gradient flipping layer, and a speaker classifier, which are connected in sequence. The prosody vector extractor is also connected to the prosody vector affine layer. The prosody vector extractor consists of a fully connected layer A, a one-dimensional convolutional layer A, a multi-head attention mechanism layer A, a fully connected layer B, and a temporal information pooling layer, which are arranged in sequence. The temporal information pooling layer is connected to the gradient flipping layer and the prosody vector affine layer respectively. The text encoder consists of a multi-head attention mechanism layer B, a speaker style adaptive normalization layer A, a one-dimensional convolution layer B, and a speaker style adaptive normalization layer B, which are arranged in this order; the multi-head attention mechanism layer B is connected to the word-to-sound conversion module; the speaker style adaptive normalization layer B is connected to the variance adapter; The mel spectrum decoder consists of a multi-head attention mechanism layer C, a speaker style adaptive normalization layer C, a one-dimensional convolutional layer C, a speaker style adaptive normalization layer D, and a fully connected layer C. The fully connected layer C is connected to the HifiGAN vocoder; the multi-head attention mechanism layer C is connected to the variance adapter; The speaker style encoder is connected to the text encoder and the mel spectrum decoder respectively; The synthetic model is trained using the "pre-training + meta-learning" model algorithm; the model parameters that can be learned by the deep learning model are θ Divided into parameters related to rhythm θ p , parameters related to the speaker θ s and other remaining parameters Composition, that is ; θ p and θ s including learnable model parameters of the speaker style encoder, Includes learnable model parameters for text encoder, mel-spectrogram decoder, and variance adapter.

2. The multi-style personalized Tibetan speech synthesis model for low-resource conditions according to claim 1 is characterized in that The model training algorithm based on "pre-training + meta-learning" includes a pre-training stage, a meta-learning stage 1, a meta-learning stage 2, and an adaptive stage; In the pre-training stage, a multi-speaker Mandarin acoustic model is pre-trained using large-scale multi-speaker Mandarin corpus. θ 0 represents the initialized acoustic model parameters, θ M1 Represents the Mandarin acoustic model parameter set after pre-training. In the pre-training stage, all learnable model parameters are Updated to ; In the meta-learning stage 1, based on the pre-trained multi-speaker Mandarin acoustic model, a small-scale multi-style multi-speaker Mandarin corpus is used to train a multi-style personalized Mandarin acoustic model using a meta-learning algorithm. θ M2 represents the multi-style personalized Mandarin acoustic model obtained in the meta-learning stage 1; In meta-learning phase 1, the parameter set is frozen θ M1 Parameters in , only update speaker related parameters Parameters related to speaker style , so that the learnable model parameters are Updated to ; In the meta-learning stage 2, based on the multi-style personalized Mandarin acoustic model, a small-scale Tibetan Mandarin corpus is used to train a multi-style personalized Tibetan acoustic model using a meta-learning algorithm. θ T represents the multi-style personalized Tibetan acoustic model obtained in the meta-learning stage 2; In meta-learning stage 2, the parameter set is frozen θ M2 Speaker style related parameters in , only update speaker related parameters and other parameters , so that the learnable model parameters are Updated to ; During the adaptation phase, freeze the parameter set θ T Rhythm-related parameters and other model parameters in ,use K The target speaker's samples are updated only by the speaker-related parameters, so that the learnable model parameters are Updated to .

Images