Tibetan language three-dialect parallel corpus data set generation method based on multi-dialect text-to-speech model

Abstract

The invention provides a Tibetan three-dialect parallel corpus data set generation method based on a multi-dialect text-to-speech model, and belongs to the field of Tibetan language processing. According to the method, a dialect ID is creatively embedded into a conditional variation auto-encoder framework, and a feed-forward network in Transform is dynamically replaced by a designed Switch ffn module, so that self-adaptive switching of dialect features is realized; and meanwhile, an improved BigVGAN vocoder is fused, so that the generated voice has acoustic characteristics of a specific dialect. Compared with a traditional method, high-quality speech generation of three dialects can be realized in a single model without depending on strictly aligned parallel corpus data, and the development cost and the data annotation cost of a multi-dialect speech synthesis system are remarkably reduced. According to the method, the overfitting problem can be effectively avoided in a low-resource scene, the generated parallel corpus is superior to the prior art in naturalness and dialect feature retention, and an effective technical solution is provided for Tibetan multi-dialect speech synthesis and language resource protection.

Description

Technical Field

[0001] The present invention belongs to the field of Tibetan language processing, and in particular relates to a method for generating parallel corpus datasets of three major Tibetan dialects based on a multi-dialect text-to-speech model. Background Art

[0002] The development of multi-dialectal Tibetan speech generation technology is rooted in the intersection of linguistics and artificial intelligence, aiming to address the complex challenges posed by the diversity of Tibetan dialects. Traditional speech synthesis techniques such as parametric synthesis and concatenative synthesis have been fundamental to this field. However, these methods rely on manually annotated pronunciation features and fixed prosodic rules, making them difficult to adapt to the significant differences between the three major Tibetan dialects: U-Tsang, Kham, and Amdo. For example, the tonal system of the U-Tsang dialect contrasts sharply with the atonal nature of the Amdo dialect, while the Kham dialect exhibits sub-dialectal phonological differentiation within itself. With breakthroughs in deep learning technology, end-to-end models such as Tacotron and WaveNet have been gradually introduced. However, these models require adaptation to the specific characteristics of the Tibetan language, such as embedding pitch contour modeling modules in tonal dialects or strengthening duration and stress prediction mechanisms in atonal dialects.

[0003] Data scarcity is another core challenge, particularly the lack of low-resource corpora such as those found in sub-dialects within Kham. This has prompted researchers to explore transfer learning and data augmentation strategies. Currently, a common approach is to pre-train the model on the U-Tsang dialect, then fine-tune the model using a small amount of data from the Kham and Amdo dialects.

[0004] At the model architecture level, cross-dialect speech generation often employs a multi-task learning framework, sharing underlying acoustic features and leveraging branching networks to capture dialect-specific pronunciation details. For example, to address the glottalization of the Amdo dialect, a pronunciation rule engine must be embedded in the front-end text analysis to ensure the accuracy of the conversion of pronunciation features. To address the tonal variations of the U-Tsang dialect, a hierarchical prosody prediction network is required to simulate intonation fluctuations. The introduction of unsupervised learning techniques has further reduced the reliance on dialect-labeled data. Extracting dialect-invariant features from speech using self-supervised models (such as wav2vec 2.0) and adapting them to small amounts of labeled data has become an emerging research direction.

[0005] Current technologies still face profound challenges in modeling cultural contexts. For example, the special expressions of religious terms or poetic rhythms are difficult to quantify using existing acoustic parameters. Cutting-edge explorations are attempting to fuse multimodal information, generating more culturally adaptable dialect variants by combining text semantics with speaker identity characteristics, or using latent variable control to achieve multi-dialect style transfer for the same text. Breakthroughs in these technologies rely not only on algorithm optimization, but also on linguists to provide fine-grained phonological rules and pragmatic knowledge, thereby improving the naturalness of synthesis while avoiding the loss of dialect uniqueness due to excessive "pan-Tibetanization" of speech. In the future, with the deepening of low-resource learning methods and interdisciplinary collaboration, Tibetan multi-dialect speech generation is expected to be more widely applied in fields such as education and cultural heritage protection. Summary of the Invention

[0006] The purpose of the present invention is to provide a method for generating parallel corpus datasets of the three major Tibetan dialects based on a multi-dialect text-to-speech model, to solve the bottlenecks of traditional methods in pronunciation diversity, prosody modeling and low-resource adaptation, and to significantly reduce the development cost of multi-dialect speech synthesis systems and the cost of parallel corpus dataset annotation.

[0007] In order to achieve the above object, the technical solution of the present invention is as follows:

[0008] A method for generating a parallel corpus dataset of three major Tibetan dialects based on a multi-dialect text-to-speech model includes the following steps:

[0009] Step 1: Input the Tibetan dialect ID, reference audio, and Tibetan text, and fuse them using a text encoder to obtain text hidden features. The text hidden features are divided into the natural logarithm of the mean and variance of the prior hidden feature distribution.

[0010] Step 2: The Mel spectrogram of the reference audio is passed through the posterior encoder to obtain hidden features;

[0011] Step 3: Map the hidden features into the Gaussian distribution space to obtain the posterior hidden features; perform hard alignment estimation on the natural logarithms of the mean and variance of the posterior hidden features and the prior hidden features to obtain the alignment matrix;

[0012] Step 4: The text hidden features are input into the duration predictor to obtain the predicted duration, which is then input into the duration discriminator along with the alignment matrix to predict the sound duration through adversarial generative training.

[0013] Step 5: Fuse the Tibetan dialect ID, reference audio, and hidden features to obtain fused features, input the fused features into the vocoder, and output the generated audio;

[0014] Step 6: Through loss function training and optimization, a trained multi-dialect text-to-speech model is obtained, and this model is used to synthesize a parallel corpus dataset of the three major Tibetan dialects.

[0015] Furthermore, the step 1 is specifically as follows:

[0016] Step 101: Encode the Tibetan dialect ID, reference audio, and Tibetan text into feature vectors of the same dimension;

[0017] Tibetan text is mapped into text features through the embedding layer, and then the context-related text features are extracted through the text encoder, and up-sampled to obtain the prior feature distribution. ;

[0018] Tibetan dialect IDs are embedded in a learnable dialect label matrix to achieve dialect-aware text encoding and obtain dialect features.

[0019] The reference audio is encoded by the convolutional neural network encoder to obtain the voiceprint features;

[0020] Step 102: Add the text features, voiceprint features, and dialect features to obtain a fusion feature of the same dimension;

[0021] Step 103: Transpose the features obtained in step S102 and obtain the text hidden features after a multi-layer self-attention mechanism , then the text hides the feature After the convolution layer, it is divided into the mean of the prior hidden feature distribution and the natural logarithm of the variance feature.

[0022] Furthermore, the vocoder is specifically a bigvgan vocoder, and the working process is as follows:

[0023] Use one-dimensional convolutional layer to fuse features To expand the dimension, multiple parallel stacked Snake modules are input for processing and then added; then the Snake1d activation function is used for activation, and the one-dimensional convolution layer is used to adjust the dimension. After the Tanh activation function, the generated audio is output.

[0024] Furthermore, the hard alignment estimation uses a monotone alignment search algorithm.

[0025] Furthermore, the self-attention mechanism includes a self-attention layer and a feedforward network structure, which splits the feedforward network into a universal feedforward network and a dialect feedforward network. The universal feedforward network is used to model the commonalities of Tibetan dialects, and the dialect feedforward network is used to model the characteristics of specific dialects. The processing results are then added and integrated, specifically including the following steps:

[0026] Step 1031: Input the output feature of the self-attention layer into the multi-head self-attention mechanism to obtain the first feature;

[0027] Step 1032: Input the first feature into a universal feed-forward network composed of a single linear layer to obtain an output common hidden feature;

[0028] Step 1033: Select a linear layer based on the dialect ID, input the first feature, and output the dialect ID feature. Specifically, there are three linear layers, corresponding to the three dialect IDs respectively.

[0029] Step 1034: Add the common hidden features and the dialect ID features to obtain the feedforward network output fusion hidden features, and the fusion hidden features output by the final layer self-attention mechanism are used as the text hidden features. .

[0030] Furthermore, the input of the Snake module specifically undergoes one-dimensional transposed convolution, upsampling, Snake1d activation function, one-dimensional downsampling, and one-dimensional dilated convolution.

[0031] Furthermore, the loss function is specifically:

[0032] ;

[0033] in, is the total loss, For the reconstruction loss, it is obtained by calculating the L1 distance between the mel spectrograms of the label audio and the generated audio; is the reconstruction loss of the posterior distribution and the prior distribution, is the loss of the duration predictor, is the least squares loss of the vocoder, where G is the generator of the vocoder; is the feature matching loss of the vocoder.

[0034] Furthermore, the reconstruction loss of the posterior distribution and the prior distribution is The calculation formula is as follows:

[0035] ;

[0036] Among them, z is the hidden feature, is the mel spectrogram of the labeled audio, is the text condition, A is the alignment matrix, represents the posterior distribution of the hidden features, Represents the prior distribution of hidden features;

[0037] The loss of the duration predictor The calculation formula is as follows:

[0038] ;

[0039] Among them, d is the duration of the pronunciation feature in the acoustic feature sequence. Indicates the alignment boundary, corresponding to the start frame position and end frame position of the pronunciation feature in time, : Indicates the duration of the pronunciation feature, is the prior duration distribution, is the posterior boundary distribution, For the posterior distribution expectations under

[0040] Least squares loss The calculation formula is as follows:

[0041] ;

[0042] in is the generator of the vocoder, is the discriminator of the vocoder, is the average value of different audio losses;

[0043] Feature matching loss The calculation formula is as follows:

[0044] ;

[0045] Where T is the total number of layers in the discriminator, For output with The discriminator of the feature number The feature map of the layer, The average of the matching losses for different audio features.

[0046] Furthermore, the method of synthesizing the parallel corpus datasets of the three major Tibetan dialects using the model specifically includes the following steps:

[0047] Step 601: The text and dialect ID are encoded separately through the embedding layer, and the reference audio is extracted through the convolution encoder to extract the voiceprint features. After the three are fused, they are encoded through the text encoder to obtain the text hidden features, and up-sampled to obtain the mean of the feature distribution. and the natural logarithm of the variance , sampling is performed on it;

[0048] Step 602: Use the duration predictor to predict the duration based on the hidden features of the input text , based on which the sampling results are upsampled to obtain the time series features , the hidden feature z is obtained by changing its feature distribution through the inverse process of normalization flow;

[0049] Step 603: Hide the feature Input into the vocoder and output is the generated speech.

[0050] Furthermore, the workflow of the trained multi-dialect text-to-speech model is as follows:

[0051] Step S601: input Tibetan text and reference audio;

[0052] Step S602: Initialize dialect id = 0;

[0053] Step S603: The Tibetan text, reference audio, and dialect ID are passed into the Tibetan text-to-speech model;

[0054] Step S604: Output the generated audio and the corresponding dialect ID;

[0055] Step S605: Determine whether id is less than 3. If so, proceed to S606; otherwise, proceed to S607;

[0056] Step S606: id = id + 1;

[0057] Step S607: End.

[0058] This paper employs a deep learning approach. Based on a conditional variational autoencoder model, this method incorporates dialect information to generate multi-dialect speech. By replacing the feedforward network module in the transformer block with a switchffn, the model switches between dialects. Drawing on the BigVGAN vocoder model, the method integrates the Tibetan dialect ID with the BigVGAN vocoder to generate speech with dialect characteristics.

[0059] The advantage of this algorithm is that it can realize speech generation of three dialects in an end-to-end model without relying on corpus-aligned speech data. It can also utilize corpus data of three dialects, solving the problem of model overfitting under low-resource corpus. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solution of the present invention, the present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0061] Figure 1 1 is a schematic diagram of the training of the Tibetan multi-dialect text-to-speech model of the present invention;

[0062] Figure 2 1 is a schematic diagram of the reasoning of the Tibetan multi-dialect text-to-speech model of the present invention;

[0063] Figure 3 This is a schematic diagram of the dialect ID fusion module of the present invention;

[0064] Figure 4 It is a schematic diagram of the switchffn module of the present invention;

[0065] Figure 5 Schematic diagram of the BigVGAN vocoder of the present invention integrating dialect ID;

[0066] Figure 6 This is a flow chart of the Tibetan multi-dialect text-to-speech model of the present invention for synthesizing parallel corpus data sets of the three major Tibetan dialects. DETAILED DESCRIPTION

[0067] In order to more clearly illustrate the technical solution of the present invention, the present invention is further described below in conjunction with the accompanying drawings and specific examples. The specific implementation of the present invention is not limited to the following examples and can be appropriately changed and modified according to actual needs without affecting the essence and scope of the present invention.

[0068] A method for generating a parallel corpus dataset of three major Tibetan dialects based on a multi-dialect text-to-speech model includes the following steps:

[0069] The first step is to train a multi-dialect text-to-speech model:

[0070] Step 1: Input the Tibetan dialect ID, reference audio, and Tibetan text, and use the text encoder to fuse them to obtain text hidden features. The text hidden features are divided into the natural logarithm of the mean and variance of the prior hidden feature distribution;

[0071] Step 101: Encode the Tibetan dialect ID, reference audio, and Tibetan text into feature vectors of the same dimension; specifically, the Tibetan text is mapped to The dense vector of dimension is used as the text feature, where batch is the batch size, length is the sentence length, and the feature length of a single Tibetan character is 192. The text encoder is then used to extract the context-related text representation and upsample it to obtain the prior feature distribution. , sampled on it;

[0072] Tibetan dialect id is mapped to The dialect features of the dimension are obtained by implementing dialect-aware text encoding through a learnable dialect label embedding matrix (U-Tsang = 0, Kham = 1, Amdo = 2);

[0073] The reference audio is obtained by encoding it through the encoder of the convolutional neural network Dimensional voiceprint features;

[0074] Step 102: Add the text features, voiceprint features and dialect features to obtain Dimension fusion features;

[0075] Step 103: Transpose the features Dimension vector, after multi-layer self-attention mechanism, the text hidden features are obtained , while hiding the text features After dimensionality increase using a one-dimensional convolutional layer, the segmentation is the mean of the prior hidden feature distribution and the natural logarithm of the variance feature.

[0076] like Figure 3 As shown in the figure, the self-attention mechanism in the text encoder can encode text features more flexibly through the self-attention layer and feedforward network structure, but it lacks the modeling function for specific dialects. Therefore, a switchffn structure is constructed to replace the feedforward network structure in the self-attention mechanism, so that the model can dynamically switch the corresponding modules according to the dialect ID; specifically, the feedforward network is split into a general feedforward network and a dialect feedforward network. The general feedforward network is used to model the common features of Tibetan dialects, and the dialect feedforward network is used to model the features of specific dialects, and then the processing results are added and integrated; the dialect ID is used to dynamically switch the dialect feedforward network, so that the model can adapt to the speech generation of the corresponding dialect while ensuring that the model inference speed remains unchanged. Figure 4 As shown, the multi-dialect switchffn module processing specifically includes the following steps:

[0077] Step 1031: Output features of the self-attention layer Input the multi-head self-attention mechanism to get the first feature ;

[0078] Step 1032: The first feature Input is fed into a feedforward network with a single linear layer as the output common hidden features ;

[0079] Step 1033: Select a linear layer based on the dialect id and transform the first feature Input, output dialect id feature ; There are 3 linear layers, corresponding to 3 dialect IDs respectively;

[0080] Step 1034: Hide the public features and dialect id features The module output fusion hidden features are added together, and the fusion hidden features output by the final layer self-attention mechanism are used as the final text hidden features. .

[0081] Step 2: Input the Mel spectrogram of the reference audio and pass it through the posterior encoder to obtain the mean and variance of the hidden features of the speech, in which the hidden features are sampled. ;

[0082] Step 3: Use the Normalized Flow module to transform the complex distribution of hidden features Mapped into Gaussian distribution space, the posterior hidden features are obtained Assuming that both the prior and posterior features conform to Gaussian distribution, the monotonic alignment search algorithm (MAS) is used to transform the posterior hidden features in the Gaussian distribution space into The distribution after the output of the text encoder with the prior and Features are hard aligned and estimated to obtain the alignment matrix , which represents the duration of each pronunciation feature;

[0083] Step 4: Use duration predictor to hide features based on input text Prediction duration , which is aligned with the matrix Input duration discriminator, and achieve sound duration prediction through adversarial generative training;

[0084] Step 5: Combine Tibetan dialect ID, reference audio and hidden features Fusion is performed to obtain fusion features , so that the generated speech has the characteristics of the corresponding dialect; the fusion features Instead of inputting the spectrogram into the vocoder, change the one-dimensional convolution dimension in the vocoder to match the hidden features Adapt and output the generated audio. Figure 5 It is a schematic diagram of the BigVGAN vocoder of the present invention that integrates the dialect ID. As shown in the figure, the process includes the following steps:

[0085] Step 501: Use a one-dimensional convolutional layer to fuse features Perform dimension expansion;

[0086] Step 502: Input the features into multiple parallel stacked Snake modules for processing and then add them together. The input of the Snake module specifically undergoes one-dimensional transposed convolution, upsampling, Snake1d activation function, one-dimensional downsampling, and one-dimensional hole convolution.

[0087] Step 503: Activate with the Snake1d activation function, adjust the dimension with a one-dimensional convolutional layer, map the value to the range of [-1, 1] with the Tanh activation function, and output the generated audio.

[0088] Step 6: Optimize the model through the loss function;

[0089] Calculate the mel spectrogram of the label audio and the generated audio separately and ,Compare and The L1 distance between them is used to obtain the reconstruction loss of the model. :

[0090] (1)

[0091] in Represents L1 distance, comparing hidden features The gap between the model and its prior distribution is the reconstruction loss of the model :

[0092] (2)

[0093] in, is the mel spectrogram of the labeled audio, is the text condition, A is the alignment matrix obtained by the MAS alignment algorithm, represents the posterior distribution of the hidden features, Represents the prior distribution of hidden features;

[0094] According to the variational autoencoder model, the loss of the duration predictor is obtained :

[0095] (3)

[0096] Among them, d is the duration of the pronunciation feature in the acoustic feature sequence. Indicates the alignment boundary, corresponding to the start frame position and end frame position of the pronunciation feature in time, : Indicates the duration of the pronunciation feature, is the prior duration distribution, is the posterior boundary distribution, For the posterior distribution The expectations below.

[0097] Compute the least squares loss for the vocoder discriminator :

[0098] (4)

[0099] in is the generator of the vocoder, is the discriminator of the vocoder, For the tag audio, To predict audio.

[0100] Compute the least squares loss for the vocoder :

[0101] (5)

[0102] in is the average value of different audio losses.

[0103] Compute feature matching loss for vocoder :

[0104] (6)

[0105] Where T is the total number of layers in the discriminator, For output with The discriminator of the feature number The feature map of the layer, The average of the matching losses for different audio features;

[0106] Add up the losses to get the total loss of model training :

[0107] (7)

[0108] By training and optimizing the model, a trained multi-dialect text-to-speech model is obtained, and the model is used to synthesize parallel corpus datasets of the three major Tibetan dialects (U-Tsang, Amdo, and Kham). The specific reasoning process is as follows Figure 2 As shown, the process includes the following steps:

[0109] S201: The text and dialect ID are encoded separately through the embedding layer, and the reference audio is extracted through the convolution encoder to obtain the voiceprint features. The three are fused to obtain the fusion feature. The fusion feature is then encoded through the text encoder and upsampled to obtain the mean of the feature distribution. and the natural logarithm of the variance , sampling is performed on it;

[0110] S202: Use the duration predictor to predict the duration based on the hidden features of the input text , based on which the sampling results are upsampled to obtain the time series features , by changing its characteristic distribution through the inverse process of normalized flow ;

[0111] S203: The input is sent to the BigVGAN vocoder, which also integrates the dialect ID and reference audio features to output the generated speech.

[0112] Figure 6 This is a flow chart of the Tibetan multi-dialect text-to-speech model of the present invention for synthesizing parallel corpus datasets of three major Tibetan dialects. As shown in the figure, the process includes the following steps:

[0113] S601: Input Tibetan text and reference audio;

[0114] S602: Initialize dialect id = 0;

[0115] S603: The text, reference audio, and dialect ID are passed to the Tibetan text-to-speech model;

[0116] S604: Output the generated audio and the corresponding dialect ID;

[0117] S605: Determine whether id is less than 3. If so, proceed to S606; otherwise, proceed to S607;

[0118] S606: id = id + 1;

[0119] S607: End.

[0120] The above descriptions of specific embodiments of the present invention are merely illustrative, and the related interface diagrams are schematic. Various modifications may be made to these embodiments in actual applications without departing from the principles and essence of the present invention. Therefore, the scope of protection of the present invention is defined by the appended claims.

Claims

1. A method for generating parallel corpus datasets of three major Tibetan dialects based on a multi-dialect text-to-speech model, characterized in that: The following steps are involved: Step 1: Input the Tibetan dialect ID, reference audio, and Tibetan text, and use the text encoder to fuse them to obtain text hidden features. The text hidden features are divided into the natural logarithm of the mean and variance of the prior hidden feature distribution; Step 2: The Mel spectrogram of the reference audio is passed through the posterior encoder to obtain hidden features; Step 3: Map the hidden features into the Gaussian distribution space to obtain the posterior hidden features; Hard align the natural logarithms of the mean and variance of the posterior hidden features and the prior hidden features to obtain the alignment matrix; Step 4: The text hidden features are input into the duration predictor to obtain the predicted duration, which is then input into the duration discriminator along with the alignment matrix to predict the sound duration through adversarial generative training. Step 5: Fuse the Tibetan dialect ID, reference audio, and hidden features to obtain fused features, input the fused features into the vocoder, and output the generated audio; Step 6: Through loss function training and optimization, a trained multi-dialect text-to-speech model is obtained, and this model is used to synthesize a parallel corpus dataset of the three major Tibetan dialects.

2. The method for generating a parallel corpus dataset of three major Tibetan dialects based on a multi-dialect text-to-speech model according to claim 1, characterized in that: The step 1 is specifically as follows: Step 101: Encode the Tibetan dialect ID, reference audio, and Tibetan text into feature vectors of the same dimension; Tibetan text is mapped into text features through the embedding layer, and then the context-related text features are extracted through the text encoder, and up-sampled to obtain the prior feature distribution. ; Tibetan dialect IDs are embedded in a learnable dialect label matrix to achieve dialect-aware text encoding and obtain dialect features. The reference audio is encoded by the convolutional neural network encoder to obtain the voiceprint features; Step 102: Add the text features, voiceprint features, and dialect features to obtain a fusion feature of the same dimension; Step 103: Transpose the features obtained in step S102 and obtain the text hidden features after a multi-layer self-attention mechanism , then the text hides the feature After the convolution layer, the segmentation is the mean of the prior hidden feature distribution and the natural logarithm of the variance feature.

3. The method for generating a parallel corpus dataset of three major Tibetan dialects based on a multi-dialect text-to-speech model according to claim 2, characterized in that: The vocoder is specifically a bigvgan vocoder, and the working process is as follows: Use one-dimensional convolutional layer to fuse features Perform dimension expansion, input multiple parallel stacked Snake modules for processing and then add them; Then the Snake1d activation function is used for activation, the one-dimensional convolution layer is used to adjust the dimension, and after the Tanh activation function, the generated audio is output.

4. The method for generating a parallel corpus dataset of three major Tibetan dialects based on a multi-dialect text-to-speech model according to claim 3, characterized in that: The hard alignment estimation uses a monotonic alignment search algorithm.

5. The method for generating a parallel corpus dataset of three major Tibetan dialects based on a multi-dialect text-to-speech model according to claim 4, characterized in that: The self-attention mechanism includes a self-attention layer and a feedforward network structure. The feedforward network is split into a universal feedforward network and a dialect feedforward network. The universal feedforward network is used to model the common features of Tibetan dialects, and the dialect feedforward network is used to model the features of specific dialects. The processing results are then added and integrated. Specifically, the following steps are included: Step 1031: Input the output feature of the self-attention layer into the multi-head self-attention mechanism to obtain the first feature; Step 1032: Input the first feature into a universal feed-forward network composed of a single linear layer to obtain an output common hidden feature; Step 1033: Select a linear layer based on the dialect ID, input the first feature, and output the dialect ID feature. Specifically, there are three linear layers, corresponding to the three dialect IDs respectively. Step 1034: Add the common hidden features and the dialect ID features to obtain the feedforward network output fusion hidden features, and the fusion hidden features output by the final layer self-attention mechanism are used as the text hidden features. .

6. The method for generating a parallel corpus dataset of three major Tibetan dialects based on a multi-dialect text-to-speech model according to claim 5, characterized in that: The input of the Snake module specifically undergoes one-dimensional transposed convolution, upsampling, Snake1d activation function, one-dimensional downsampling, and one-dimensional hole convolution.

7. The method for generating a parallel corpus dataset of three major Tibetan dialects based on a multi-dialect text-to-speech model according to claim 6, characterized in that: The loss function is specifically: ； in, is the total loss, For the reconstruction loss, it is obtained by calculating the L1 distance between the mel spectrograms of the label audio and the generated audio; is the reconstruction loss of the posterior distribution and the prior distribution, is the loss of the duration predictor, is the least squares loss of the vocoder, where G is the generator of the vocoder; is the feature matching loss of the vocoder.

8. The method for generating a parallel corpus dataset of three major Tibetan dialects based on a multi-dialect text-to-speech model according to claim 7, characterized in that: The reconstruction loss of the posterior distribution and the prior distribution The calculation formula is as follows: ； Among them, z is the hidden feature, is the Mel spectrogram of the labeled audio, is the text condition, A is the alignment matrix, represents the posterior distribution of the hidden features, Represents the prior distribution of hidden features; The loss of the duration predictor The calculation formula is as follows: ； Among them, d is the duration of the pronunciation feature in the acoustic feature sequence. Indicates the alignment boundary, corresponding to the start frame position and end frame position of the pronunciation feature in time, : Indicates the duration of the pronunciation feature, is the prior duration distribution, is the posterior boundary distribution, For the posterior distribution expectations under Least squares loss The calculation formula is as follows: ； in is the generator of the vocoder, is the discriminator of the vocoder, is the average value of different audio losses; Feature matching loss The calculation formula is as follows: ； Where T is the total number of layers in the discriminator, For output with The discriminator of the feature number The feature map of the layer, The average of the matching losses for different audio features.

9. The method for generating a parallel corpus dataset of three major Tibetan dialects based on a multi-dialect text-to-speech model according to claim 8, characterized in that: The method of synthesizing the parallel corpus datasets of the three major Tibetan dialects using the model specifically includes the following steps: Step 601: The text and dialect ID are encoded separately through the embedding layer, and the reference audio is extracted through the convolution encoder to extract the voiceprint features. After the three are fused, they are encoded through the text encoder to obtain the text hidden features, and up-sampled to obtain the mean of the feature distribution. and the natural logarithm of the variance , sampling is performed on it; Step 602: Use the duration predictor to predict the duration based on the hidden features of the input text , based on which the sampling results are upsampled to obtain the time series features , the hidden feature z is obtained by changing its feature distribution through the inverse process of normalization flow; Step 603: Hide the feature Input into the vocoder and output is the generated speech.

10. The method for generating parallel corpus datasets of three major Tibetan dialects based on a multi-dialect text-to-speech model according to claim 9, characterized in that: The workflow of the trained multi-dialect text-to-speech model is as follows: Step S601: input Tibetan text and reference audio; Step S602: Initialize dialect id = 0; Step S603: The Tibetan text, reference audio, and dialect ID are passed into the Tibetan text-to-speech model; Step S604: Output the generated audio and the corresponding dialect ID; Step S605: Determine whether id is less than 3. If so, proceed to S606; otherwise, proceed to S607; Step S606: id = id + 1; Step S607: End.

Images