Prosodic modeling method and modeling system integrating Amdo Tibetan phoneme vectors

Abstract

The present invention proposes a prosody modeling method and a modeling system integrating Ando Tibetan phoneme vectors. The method obtains Ando Tibetan texts and forms a corpus with prosody annotations according to syllables from the Ando Tibetan texts; converts the corpus with prosody annotations into Ando Tibetan phonemes through the Ando-Wyley transliteration rule; calculates Ando Tibetan factors using a convolutional neural network to obtain Ando Tibetan phoneme vectors; inputs the Ando Tibetan phoneme vectors into a BiLSTM network to obtain prediction labels of different prosodys; and optimizes the prediction labels of different prosodys using a cross entropy loss function; thereby making the synthesized Ando Tibetan speech have more prosody information, improving the naturalness of Ando Tibetan speech synthesis, and making the sound quality sound more natural, thereby facilitating the development of intelligent speech information processing in Tibetan areas.

Description

Technical Field

[0001] The present invention relates to the technical field of speech synthesis, and in particular to a prosody modeling method and a modeling system integrating Ando Tibetan phoneme vectors. Background Art

[0002] Speech synthesis technology is an important part of human-computer interaction and can be widely used in voice navigation, smart home, virtual anchor, teaching robot, etc. Tibetan speech synthesis is limited by the small size of corpus data, high data annotation cost, small research scale, etc., and its progress is slower than that of English and Chinese. Since the tone information of Amdo Tibetan is not obvious, the rhythm information of Amdo Tibetan speech synthesis is not obvious.

[0003] The problem that speech synthesis solves is to convert text into human-understandable speech information. The goal of speech synthesis is to improve the intelligibility and naturalness of synthesized audio. Intelligibility means that the synthesized audio can be understood by people, and naturalness means that the synthesized audio is more like the sound made by humans. Speech synthesis is mainly divided into two tasks: front-end and back-end: 1. The front-end task refers to the processing of linguistic information of the text and text analysis, mainly sentence segmentation, word segmentation, normalization, part-of-speech analysis and rhythm prediction of the text; 2. The back-end task refers to the processing of acoustic information, mainly improving the vocoder to improve the quality of synthesized speech. How to improve the front-end task of text in Tibetan speech synthesis to improve the rhythmic effect of Amdo Tibetan speech synthesis is a technical problem that needs to be solved at present. Summary of the invention

[0004] The object of the present invention is to solve at least one of the above-mentioned technical drawbacks.

[0005] To this end, one purpose of the present invention is to propose a rhythmic modeling method and modeling system that integrates Ando Tibetan phoneme vectors, so that the synthesized Ando Tibetan speech has more rhythmic information, improves the naturalness of Ando Tibetan speech synthesis, and makes the sound quality sound more natural, which is conducive to the development of intelligent speech information processing in Tibetan areas.

[0006] In order to achieve the above purpose, a prosody modeling method integrating Amdo Tibetan phoneme vectors includes the following steps:

[0007] S1. Obtaining an Amdo Tibetan text, and converting the Amdo Tibetan text into a corpus with rhythmic annotations based on syllables;

[0008] S2. Convert the corpus with prosodic annotations into Ando Tibetan phonemes using the Ando-Willie transliteration rule; calculate the Ando Tibetan factors using a convolutional neural network to obtain the Ando Tibetan phoneme vector;

[0009] S3, input the Amdo Tibetan phoneme vector into the BiLSTM network to obtain the prediction labels of different prosody;

[0010] S4. Use the cross entropy loss function to optimize the predicted labels of different rhythms.

[0011] Further, preferably, in S1, the step of forming the Amdo Tibetan text into a corpus that is rhythmically annotated according to syllables comprises the following method:

[0012] Acquire Amdo Tibetan audio data and corresponding Tibetan text information, and divide the Tibetan text information into three structures: prosodic words, prosodic phrases, and intonation phrases;

[0013] Extracting syllables from the divided prosodic words, prosodic phrases or intonation phrases respectively, and drawing a pitch distribution diagram of the extracted syllables;

[0014] The stress probability of the syllable is calculated according to the pitch distribution map, and the extracted syllable is marked according to the stress probability to form a corpus with prosodic annotations.

[0015] Further, preferably, in S2, the corpus with prosodic annotations is passed through the Ando-Willi transcription module to obtain the Ando Tibetan phoneme sequence; the Ando Tibetan phoneme sequence is taken as input and put into the convolutional neural network, and after passing through 8 convolutional layers and 8 pooling layers, it is mapped to the output space through the dense layer of the residual neural network to obtain the Ando Tibetan phoneme vector.

[0016] Further, preferably, in S3, the step of inputting the Ando Tibetan phoneme vector into the BiLSTM network to obtain prediction labels of different prosody includes:

[0017] The Ando Tibetan phoneme vector is placed in the BiLSTM network framework, the function is optimized by Adam, the discard rate is set to 0.5, and the multi-classification problem is solved through the Softmax layer to obtain labels for different rhythm predictions.

[0018] Further, preferably, in S4, a cross entropy loss function is used to optimize the predicted labels of different rhythms, including:

[0019] According to the types of different prosody prediction labels obtained, the samples of the prediction labels are multi-classified, and the cross entropy loss value is calculated according to the probability of each prediction sample using the following formula;

[0020]

[0021] Among them, y i,k Indicates that the true label of the i-th sample is k, there are K label values ​​and N samples, p i,k It represents the probability that the i-th sample is predicted to be the k-th label value.

[0022] The present invention also provides a prosody modeling system integrating Ando Tibetan phoneme vectors, including a data acquisition module, a prosody prediction model and an optimization module;

[0023] The data acquisition module is used to acquire the Amdo Tibetan text and form the Amdo Tibetan text into a corpus that is rhythmically annotated according to syllables;

[0024] The prosody prediction model is used to convert the corpus with prosody annotations into Ando Tibetan phonemes through the Ando-Willi transliteration rule; calculate the Ando Tibetan factors using a convolutional neural network to obtain an Ando Tibetan phoneme vector; input the Ando Tibetan phoneme vector into a BiLSTM network to obtain prediction labels of different prosody;

[0025] The optimization module optimizes the prediction labels of different rhythms using a cross entropy loss function.

[0026] Further, preferably, the data acquisition module further comprises acquiring Amdo Tibetan audio data and corresponding Tibetan text information, and dividing the Tibetan text information into three structures: prosodic words, prosodic phrases, and intonation phrases;

[0027] Extracting syllables from the divided prosodic words, prosodic phrases or intonation phrases respectively, and drawing a pitch distribution diagram of the extracted syllables;

[0028] The stress probability of the syllable is calculated according to the pitch distribution map, and the extracted syllable is marked according to the stress probability to form a corpus with prosodic annotations.

[0029] Further, preferably, the rhythm prediction model includes an Andoweili transcription module, a first convolutional pooling layer, a second convolutional pooling layer and a BiLSTM network;

[0030] The Ando Weili transcription module is used to convert the input Tibetan text into an Ando Tibetan phoneme sequence;

[0031] The first convolutional pooling layer is used to obtain an Anduo Tibetan phoneme vector by performing convolution kernel and pooling kernel operations on the input Anduo Tibetan phoneme sequence;

[0032] The BiLSTM network is used to optimize and learn the input Ando Tibetan phoneme vector to obtain labels of different prosody predictions.

[0033] Further, preferably, the data processing module also includes taking the Ando Tibetan phoneme sequence as input, putting it into a convolutional neural network, passing through 8 convolutional layers and 8 pooling layers, and mapping it to the output space through a dense layer of a residual neural network to obtain an Ando Tibetan phoneme vector.

[0034] Further, preferably, the evaluation and optimization module performs multi-classification on samples of the prediction labels according to the types of different prosody prediction labels obtained, and calculates the cross entropy loss value according to the probability of each prediction sample using the following formula;

[0035]

[0036] Among them, y i,k Indicates that the true label of the i-th sample is k, there are K label values ​​and N samples, p i,k It represents the probability that the i-th sample is predicted to be the k-th label value.

[0037] A prosody modeling method and modeling system integrating Amdo Tibetan phoneme vectors provided according to an embodiment of the present invention has at least the following advantages compared with the prior art:

[0038] 1. The method of integrating Amdo Tibetan morphology and phonemes was used. In the front end of Amdo Tibetan speech synthesis, the phoneme vector was input into the neural network as a representation vector, so as to obtain more language information of Amdo Tibetan and reduce the information sparsity problem caused by the word vector method;

[0039] 2. Pre-training is performed using Chinese corpus with a large amount of annotated data. When annotating the corpus, a two-level annotation method is used. The pitch distribution of the rhythmic words and rhythmic phrases annotated for the first time is statistically analyzed, and the syllables are annotated according to the pitch probability to form a rhythmically annotated corpus. Since the syllable is composed of the smallest pronunciation unit - the phoneme, the phoneme vector is formed by the syllable; when predicting the rhythm, the rhythm classification can be obtained more accurately, while avoiding the subjectivity of manual annotation or the omission of Tibetan abbreviations.

[0040] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0042] Figure 1 A flow chart of the prosody modeling method integrating Amdo Tibetan phoneme vectors provided by the present invention;

[0043] Figure 2 A schematic diagram of a prosody prediction model in a prosody modeling system integrating Anduo Tibetan phoneme vectors provided by the present invention; DETAILED DESCRIPTION

[0044] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0045] like Figure 1 As shown, a prosody modeling method integrating Amdo Tibetan phoneme vectors according to an embodiment of the present invention comprises the following steps:

[0046] S1. Obtaining an Amdo Tibetan text, and converting the Amdo Tibetan text into a corpus with rhythmic annotations based on syllables;

[0047] S2. Convert the corpus with prosodic annotations into Ando Tibetan phonemes using the Ando-Willie transliteration rule; calculate the Ando Tibetan factors using a convolutional neural network to obtain the Ando Tibetan phoneme vector;

[0048] S3, input the Amdo Tibetan phoneme vector into the BiLSTM network to obtain the prediction labels of different prosody;

[0049] S4. Use the cross entropy loss function to optimize the predicted labels of different rhythms.

[0050] In S1, the step of forming a corpus with prosody annotations based on syllables from the Amdo Tibetan text includes the following method:

[0051] Acquire Amdo Tibetan audio data and corresponding Tibetan text information, and divide the Tibetan text information into three structures: prosodic words, prosodic phrases, and intonation phrases;

[0052] Extracting syllables from the divided prosodic words, prosodic phrases or intonation phrases respectively, and drawing a pitch distribution diagram of the extracted syllables;

[0053] The stress probability of the syllable is calculated according to the pitch distribution map, and the extracted syllable is marked according to the stress probability to form a corpus with prosodic annotations.

[0054] Among them, when obtaining the Amdo Tibetan audio data, a self-designed and recorded Amdo Tibetan speech synthesis dataset was used. The dataset contains two speakers, one male and one female, aged between 20 and 30 years old, who have lived in the Amdo area of ​​Qinghai Province for a long time. Both speakers are professional hosts of Xining Radio and Television Station in Qinghai Province, with positive and intellectual voices. Professional instruments and equipment are used in the audio recording process, and the recording signal-to-noise ratio is not less than 35dB. The dataset has a total of 11,796 audio data, with a sampling rate of 48KHz, uncompressed PCM WAV format, and an amplitude of -30,000 to +30,000. The 11,796 audio texts cover multiple fields such as Tibetan novels, music, and news. After phoneme balance processing, it ensures that the text corpus can contain more voice information and secondary voice information.

[0055] First, the experimental text data was proofread, and Tibetan students in the Amdo area checked 11,796 Amdo Tibetan data to verify the correspondence between audio and text. Next, the silent segments were removed, and then the corpus was screened to select longer sentences with rich rhythmic information. Finally, the corpus was annotated according to the rhythmic relationship of Tibetan. According to the rhythmic structure of Chinese, Tibetan is also divided into three structures: rhythmic words, rhythmic phrases, and intonation phrases. Define #1 as a rhythmic word, #2 as a rhythmic phrase structure, and #3 as an intonation phrase. According to the rhythmic structure of Tibetan, the labels #1, #2, and #3 are added to the corpus, for example: According to the structure of the preliminary division of Tibetan, the pitch distribution diagrams are drawn in the corpus formed after adding labels, with syllables as the smallest unit, and the stress probability of syllables is calculated according to the pitch distribution diagrams. The extracted syllables are marked according to the stress probability to form a corpus with rhythmic annotations. After preprocessing, 80% of the text is used as a training set, 10% as a test set, and 10% as a validation set.

[0056] In S2, the corpus with prosodic annotations is passed through the Ando-Willi transliteration module to obtain the Ando Tibetan phoneme sequence; the Ando Tibetan phoneme sequence is used as input and put into the convolutional neural network. After 8 layers of convolutional layers and 8 layers of pooling layers, it is mapped to the output space through the dense layer of the residual neural network to obtain the Ando Tibetan phoneme vector. Phonemes are very important pronunciation units in Tibetan. Since the Tibetan character sequence is diverse and complex, and the pronunciation of Tibetan is closely related to phonemes, it is more advantageous to use phonemes to represent the pronunciation information of Tibetan. Usually, a phoneme is the smallest pronunciation segment, and a syllable is composed of multiple phonemes. The syllable can well reflect the pronunciation and can better distinguish different word meanings. Syllables are produced by the tension of the muscles of the pronunciation organs. Syllables usually contain multiple vowels or compound vowels, and even some languages ​​also contain consonants. However, the central structure of the syllable is still vowels. The Tibetan syllable structure is usually divided into open syllables and closed syllables, which is similar to other languages. When different syllables form sentences, different phonetic phenomena will occur, which is also related to the naturalness and intelligibility of Tibetan speech synthesis.

[0057] In S3, the Ando Tibetan phoneme vector is input into the BiLSTM network to obtain prediction labels of different prosody, including:

[0058] The Ando Tibetan phoneme vector is placed in the BiLSTM network framework, the function is optimized by Adam, the discard rate is set to 0.5, and the multi-classification problem is solved through the Softmax layer to obtain labels for different rhythm predictions.

[0059] In a specific embodiment, the network of the present invention is based on BiLSTM (bidirectional long short-term memory network) and multi-layer convolutional network. First, the input is Ando Tibetan text, which is converted into Ando Tibetan phonemes through Ando-Willie transliteration rules. When passing through the first layer of convolutional network C1, 8 convolution kernels of size 6*6 are first passed to obtain 8 feature maps, and through the first layer of pooling layer, 2*2 cores are used to perform pooling operations, and then the output features are put into the second convolutional network C2, and the convolution kernel size is 4, and the convolution with a step length of 1 is connected to the same order features. Then through the second layer of pooling layer, 2*2 cores are also used to perform pooling operations to obtain Ando Tibetan phoneme vectors, which are put into the BiLSTM network framework, which is composed of 2 layers of LSTM layers, and each direction contains 180 neurons. This network framework is optimized by Adam to function, and the learning rate is 0.0001, and Dropoutrate is set to 0.5 to prevent overfitting. Finally, through the Softmax layer, it is used to solve the multi-classification problem well, and the labels of different rhythm predictions are obtained.

[0060] In S4, the cross entropy loss function is used to optimize the predicted labels of different rhythms, including:

[0061] According to the types of different prosody prediction labels obtained, the samples of the prediction labels are multi-classified, and the cross entropy loss value is calculated according to the probability of each prediction sample using the following formula;

[0062]

[0063] Among them, y i,k Indicates that the true label of the i-th sample is k, there are K label values ​​and N samples, p i,k It represents the probability that the i-th sample is predicted to be the k-th label value.

[0064] Furthermore, for the loss value calculated by the cross entropy loss function, likelihood estimation is performed according to the set loss threshold. If the calculated loss value is large, it indicates that the classification result of the sample is unclear. At this time, the maximum likelihood estimate of the sample is calculated, and the maximum likelihood estimate is used to correct the cross entropy loss of the sample. The cross entropy function can be used to continuously optimize the rhythm prediction model of the Tibetan phoneme vector proposed in this paper, thereby continuously improving the effect of rhythm prediction.

[0065] The present invention also provides a prosody modeling system integrating Ando Tibetan phoneme vectors, including a data acquisition module, a prosody prediction model and an optimization module;

[0066] The data acquisition module is used to acquire the Amdo Tibetan text and form the Amdo Tibetan text into a corpus that is rhythmically annotated according to syllables;

[0067] The prosody prediction model is used to convert the corpus with prosody annotations into Ando Tibetan phonemes through the Ando-Willi transliteration rule; calculate the Ando Tibetan factors using a convolutional neural network to obtain an Ando Tibetan phoneme vector; input the Ando Tibetan phoneme vector into a BiLSTM network to obtain prediction labels of different prosody;

[0068] The optimization module optimizes the prediction labels of different rhythms using a cross entropy loss function.

[0069] Further, preferably, the data acquisition module further comprises acquiring Amdo Tibetan audio data and corresponding Tibetan text information, and dividing the Tibetan text information into three structures: prosodic words, prosodic phrases, and intonation phrases;

[0070] Extracting syllables from the divided prosodic words, prosodic phrases or intonation phrases respectively, and drawing a pitch distribution diagram of the extracted syllables;

[0071] The stress probability of the syllable is calculated according to the pitch distribution map, and the extracted syllable is marked according to the stress probability to form a corpus with prosodic annotations.

[0072] Further, preferably, the rhythm prediction model includes an Andoweili transcription module, a first convolutional pooling layer, a second convolutional pooling layer and a BiLSTM network;

[0073] The Ando Weili transcription module is used to convert the input Tibetan text into an Ando Tibetan phoneme sequence;

[0074] The first convolutional pooling layer is used to obtain an Anduo Tibetan phoneme vector by performing convolution kernel and pooling kernel operations on the input Anduo Tibetan phoneme sequence;

[0075] The BiLSTM network is used to optimize and learn the input Ando Tibetan phoneme vector to obtain labels of different prosody predictions.

[0076] Further, preferably, the data processing module also includes taking the Ando Tibetan phoneme sequence as input, putting it into a convolutional neural network, passing through 8 convolutional layers and 8 pooling layers, and mapping it to the output space through a dense layer of a residual neural network to obtain an Ando Tibetan phoneme vector.

[0077] Further, preferably, the evaluation and optimization module performs multi-classification on samples of the prediction labels according to the types of different prosody prediction labels obtained, and calculates the cross entropy loss value according to the probability of each prediction sample using the following formula;

[0078]

[0079] Among them, y i,k Indicates that the true label of the i-th sample is k, there are K label values ​​and N samples, p i,k It represents the probability that the i-th sample is predicted to be the k-th label value.

[0080] During the model training process, the parameters were continuously adjusted and the optimal parameters were selected as the final experimental parameters. The final batch size parameter was set to 50 and the epoch was set to 100.

[0081] Evaluation method:

[0082] In the experiment of statistically analyzing the error rate of word internal division, this chapter randomly selected 40 error text messages from the baseline experiment and the proposed method experiment, with a total of 432 words. The text in the error text message remains consistent, but the difference lies in the different prosodic boundary divisions. This paper manually calculates the error rate of word internal division in the 40 error texts. The experimental results are shown in Table 4-4.

[0083] Table 4-4 Word segmentation error rate statistics

[0084]

[0085] In Table 4-4, it can be seen that the error rate of the Tibetan phoneme vector modeling method proposed in this chapter is 6.8% lower than that of the baseline method. Therefore, the prosody modeling method based on Tibetan phoneme vectors proposed in this application is effective.

[0086] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0087] Although the embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and cannot be understood as limiting the present invention. Those skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention without departing from the principles and purpose of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A prosody modeling method integrating Amdo Tibetan phoneme vectors, characterized in that: The following steps are involved: S1. Obtaining an Amdo Tibetan text, and forming the Amdo Tibetan text into a corpus with prosodic annotations according to syllables; including: obtaining Amdo Tibetan audio data and corresponding Tibetan text information, and dividing the Tibetan text information into three structures: prosodic words, prosodic phrases, and intonation phrases; Extracting syllables from the divided prosodic words, prosodic phrases or intonation phrases respectively, and drawing a pitch distribution diagram of the extracted syllables; Calculate the stress probability of the syllable according to the pitch distribution map, and mark the extracted syllables according to the stress probability to form a corpus with prosodic annotations S2. Convert the corpus with prosodic annotations into Ando Tibetan phonemes through the Ando-Willie transliteration rule; calculate the Ando Tibetan factors using a convolutional neural network to obtain an Ando Tibetan phoneme vector; use the Ando-Willie transliteration module to obtain an Ando Tibetan phoneme sequence from the corpus with prosodic annotations; use the Ando Tibetan phoneme sequence as input, put it into the convolutional neural network, pass through 8 convolutional layers and 8 pooling layers, and then map it to the output space through the dense layer of the residual neural network to obtain an Ando Tibetan phoneme vector; S3, input the Amdo Tibetan phoneme vector into the BiLSTM network to obtain the prediction labels of different prosody; S4. Use the cross entropy loss function to optimize the predicted labels of different rhythms.

2. The prosody modeling method integrating Amdo Tibetan phoneme vectors according to claim 1 is characterized in that: In S3, the Ando Tibetan phoneme vector is input into the BiLSTM network to obtain prediction labels of different prosody, including: The Ando Tibetan phoneme vector is placed in the BiLSTM network framework, the function is optimized by Adam, the discard rate is set to 0.5, and the multi-classification problem is solved through the Softmax layer to obtain labels for different rhythm predictions.

3. The prosody modeling method integrating Amdo Tibetan phoneme vectors according to claim 1 is characterized in that: exist In S4, the cross entropy loss function is used to optimize the predicted labels of different rhythms, including: According to the types of different prosody prediction labels obtained, the samples of the prediction labels are multi-classified, and the cross entropy loss value is calculated according to the probability of each prediction sample using the following formula; Among them, y i,k Indicates that the true label of the i-th sample is k, there are K label values ​​and N samples, p i,k It represents the probability that the i-th sample is predicted to be the k-th label value.

4. A prosody modeling system integrating Amdo Tibetan phoneme vectors, characterized in that: It includes data acquisition module, rhythm prediction model and optimization module; The data acquisition module is used to acquire the Amdo Tibetan text and form the Amdo Tibetan text into a corpus that is rhythmically annotated according to syllables; including: acquiring the Amdo Tibetan audio data and the corresponding Tibetan text information, and dividing the Tibetan text information into three structures: prosodic words, prosodic phrases, and intonation phrases; Extracting syllables from the divided prosodic words, prosodic phrases or intonation phrases respectively, and drawing a pitch distribution diagram of the extracted syllables; Calculate the stress probability of the syllable according to the pitch distribution map, and mark the extracted syllable according to the stress probability to form a corpus with prosodic annotations; The prosody prediction model is used to convert the corpus with prosody annotations into Ando Tibetan phonemes through the Ando-Willi transliteration rule; calculate the Ando Tibetan factors using a convolutional neural network to obtain an Ando Tibetan phoneme vector; including: taking the Ando Tibetan phoneme sequence as input, putting it into the convolutional neural network, passing through 8 convolutional layers and 8 pooling layers, and mapping it to the output space through the dense layer of the residual neural network to obtain the Ando Tibetan phoneme vector; inputting the Ando Tibetan phoneme vector into the BiLSTM network to obtain prediction labels of different prosody; The optimization module optimizes the prediction labels of different rhythms using a cross entropy loss function.

5. The prosody modeling system integrating Amdo Tibetan phoneme vectors according to claim 4 is characterized in that: The rhythm prediction model includes an Andoweili transcription module, a first convolutional pooling layer, a second convolutional pooling layer and a BiLSTM network; The Ando Weili transcription module is used to convert the input Tibetan text into an Ando Tibetan phoneme sequence; The first convolutional pooling layer is used to obtain an Anduo Tibetan phoneme vector by performing convolution kernel and pooling kernel operations on the input Anduo Tibetan phoneme sequence; The BiLSTM network is used to optimize and learn the input Ando Tibetan phoneme vector to obtain labels of different prosody predictions.

6. The prosody modeling system integrating Amdo Tibetan phoneme vectors according to claim 4 is characterized in that: The optimization module performs multi-classification on samples of the prediction labels according to the types of different prosody prediction labels obtained, and calculates the cross entropy loss value according to the probability of each prediction sample using the following formula; Among them, y i,k Indicates that the true label of the i-th sample is k, there are K label values ​​and N samples, p i,k It represents the probability that the i-th sample is predicted to be the k-th label value.

Images