Speech conversion method from Tibetan Anduodialect to defense Tibetan dialect

Abstract

The invention discloses a speech conversion method from Tibetan Andor dialects to defense dialects, which comprises the following steps: S1, establishing a corpus: establishing a Tibetan text corpus, and recording a speech corpus parallel to the Tibetan Andor dialects and the defense dialects according to the corpus; s2, acoustic feature analysis and processing: carrying out fundamental frequency analysis and duration analysis on single-character and double-character words of an Anduo dialect and a defense and Tibetan dialect, finding out conversion parameters, extracting formant tracks and consonant transition acoustic features in the analysis process, and analyzing phoneme combinations of resonants and consonants; s3, establishing a conversion model: establishing a single-character conversion model and a double-character word conversion model from an Andol dialect to a defense dialect, and constructing a vowel generation model for a long vowel and a nasal vowel in the defense dialect; and S4, voice conversion: performing voice conversion from an Andol dialect to a defense dialect by using a single-character conversion model, a double-character word conversion model and a vowel generation model.

Description

Technical Field

[0001] The present invention relates to the technical field of speech data processing, and in particular to a method for speech conversion from the Tibetan Amdo dialect to the U-Tsang dialect. Background Art

[0002] Tibetans have an ancient history and culture, so they also have their own language for internal communication, namely Tibetan, which is a relatively large minority language among ethnic minorities. Tibetan is divided into the U-Tsang dialect, the Kang dialect and the Amdo dialect. Although these three dialects all use Tibetan as a written language, there is a certain difference in the pronunciation of the three dialects. With the development of science and technology, speech conversion technology is being used more and more widely in language learning and communication. For Tibetan, realizing speech conversion between different dialects has important application value.

[0003] After searching, the Chinese patent number CN118262699A discloses a method for voice conversion from the Tibetan Amdo dialect to the U-Tsang dialect, which includes the following steps: Step 1: Establish a parallel corpus of the Tibetan Amdo dialect and the U-Tsang dialect; Step 2: Perform acoustic feature analysis on the Tibetan Amdo dialect and the U-Tsang dialect respectively; Step 3: Establish a conversion model using the five-degree character tone model; Step 4: Perform voice conversion from the Tibetan Amdo dialect to the U-Tsang dialect using the STRANGHT algorithm. This technical solution is used to solve the problem of difficulty in conversion between the existing Tibetan Amdo dialect and the U-Tsang dialect.

[0004] During the use of the above system, the U-Tsang dialect has long vowels and nasal vowels, which may last for a long time in pronunciation and are accompanied by nasal resonance. The Amdo dialect does not have long vowels and nasal vowels, and its vowel system may be shorter and more direct. Since there are no long vowels and nasal vowels in the Amdo dialect, the conversion model may encounter difficulties in generating these phonemes, resulting in unnatural or missing vowels in the converted speech. Therefore, a speech conversion method from the Tibetan Amdo dialect to the U-Tsang dialect is proposed. Summary of the invention

[0005] The purpose of the present invention is to solve the shortcomings of the prior art that, since there are no long vowels and nasalized vowels in the Amdo dialect, the conversion model may encounter difficulties in generating these phonemes, resulting in unnatural or missing vowels in the converted speech, and a method for speech conversion from the Tibetan Amdo dialect to the U-Tsang dialect is proposed.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A method for converting the speech of the Tibetan Amdo dialect to the U-Tsang dialect comprises the following steps:

[0008] S1: Establish corpus: Establish a Tibetan text corpus, and record the parallel speech corpus of the Tibetan Amdo dialect and U-Tsang dialect based on the corpus. Segment, annotate and establish a catalog index for the speech corpus of the Amdo dialect and U-Tsang dialect;

[0009] S2: Acoustic feature analysis and processing: Perform fundamental frequency analysis and duration analysis on single-word and double-word words of Amdo dialect and U-Tsang dialect, compare the differences in fundamental frequency and duration between the two dialects, find out the key parameters of conversion, extract the resonance peak trajectory and consonant transition acoustic features during the analysis, and analyze the phoneme combination of complex consonants. The resonance peak refers to some areas where energy is relatively concentrated in the spectrum of sound. These areas reflect the physical characteristics of the vocal tract (resonance cavity). In the phoneme combination of complex consonants, the resonance peak trajectory of different phonemes can reflect the changes in the shape and size of the vocal tract, which helps to analyze the combination mode and pronunciation characteristics of phonemes. Resonance peak is an important factor in determining sound quality and timbre. In complex consonants, the resonance peaks of different phonemes have different distribution positions and characteristics. Consonant transition acoustic features refer to the acoustic changes produced when consonant phonemes are connected. These changes include changes in frequency, amplitude, timbre, etc. In complex consonants, the transition acoustic features between consonants can reveal the connection relationship and pronunciation characteristics between them, which helps to analyze the structure and pronunciation rules of complex consonants;

[0010] S3: Conversion model establishment: using the five-degree character tone model, combined with LSTM, a single-word conversion model and a two-word word conversion model from the Amdo dialect to the U-Tsang dialect are established. The single-word conversion model and the two-word word conversion model process the generation and conversion of tone information. For the long vowels and nasalized vowels in the U-Tsang dialect, a vowel generation model is constructed. The vowel generation model is based on the input Amdo dialect vowels, and generates long vowels and nasalized vowels that meet the characteristics of the U-Tsang dialect by adjusting the duration, sound intensity and nasal resonance of the vowels;

[0011] S4: Speech conversion: Using the single-word conversion model, the two-word word conversion model and the vowel generation model, the speech conversion from Amdo dialect to U-Tsang dialect is performed through the STRAI GHT algorithm conversion algorithm. In the speech conversion, long vowels and nasalized vowels are generated by synthesis.

[0012] The above further includes:

[0013] Further in S1, the specific steps of establishing the corpus are:

[0014] Build a Tibetan text corpus:

[0015] Collecting text data: Collecting a large amount of Tibetan texts from various Tibetan publications, online resources, academic papers, etc., ensuring that the texts cover a wide range of topics and contexts to reflect the diversity of the language;

[0016] Text preprocessing: clean the collected text, remove irrelevant characters (such as garbled characters, special symbols, etc.), perform word segmentation, split the text into words or phrases, and mark the parts of speech, providing a basis for the subsequent speech corpus construction;

[0017] Parallel speech corpora of Amdo and U-Tsang Tibetan dialects:

[0018] Selection of speakers: Select speakers who are proficient in the Amdo and U-Tsang dialects of Tibetan, ensuring that the speakers are representative and can reflect the characteristics of the dialects;

[0019] Recording speech: Use high-quality recording equipment to record the speaker reading sentences in the text corpus in a quiet environment. Record each sentence multiple times to ensure the clarity and accuracy of the speech.

[0020] Segment, annotate and create a catalog index for the phonetic corpus of the Amdo dialect and the U-Tsang dialect:

[0021] Speech segmentation: Segment the recorded speech into sentences or phrases, and align the speech with the corresponding text using speech recognition software or manual annotation;

[0022] Annotation: Annotate each speech segment, including information such as phonemes, syllables, and tones. Special phonemes such as complex consonants, long vowels, and nasalized vowels require special annotation;

[0023] Create a directory index: Create a unique identifier for each voice segment, create an index table, and associate the identifier with the corresponding text, voice segment, and annotation information.

[0024] Furthermore, in S2, the specific steps of the fundamental frequency analysis are:

[0025] Fundamental frequency extraction: The fundamental frequency information of the speech sample is extracted through the autocorrelation function. The fundamental frequency is usually defined as the lowest frequency component of the sound and is the main tone of the sound. The autocorrelation function is expressed as:

[0026]

[0027] Where x(n) is the speech signal, τ is the delay time, and the fundamental frequency corresponds to the delay time that makes the autocorrelation function reach the maximum value;

[0028] Frequency comparison: compare the frequency values ​​of the same single-word and double-word words between the Amdo dialect and the U-Tsang dialect, and analyze the differences in frequency between the two dialects, such as the mean value, standard deviation and other statistics;

[0029] Key parameter extraction: Based on the fundamental frequency comparison results, the key parameters that affect the fundamental frequency difference between the two dialects are extracted. The key parameters include the average value, range and change trend of the fundamental frequency.

[0030] Furthermore, in S2, the specific steps of the duration analysis are:

[0031] Duration extraction: Extract the duration information of the speech sample. Duration is usually defined as the length of time from the beginning to the end of the speech sample. Duration is expressed as:

[0032] T=t end -t start

[0033] Among them, t start and t end are the start and end time of the speech signal respectively;

[0034] Duration comparison: compare the duration values ​​of the same single-word and double-word words in the Amdo dialect and the U-Tsang dialect, and analyze the differences in duration between the two dialects, such as the mean value, standard deviation and other statistics;

[0035] Key parameter extraction: Based on the duration comparison results, the key parameters that affect the duration difference between the two dialects are extracted. The key parameters include the average value, range and change trend of the duration.

[0036] Further, in S2, the specific steps of extracting and analyzing the formant trajectory and consonant transition acoustic features;

[0037] Formant trajectory extraction: Perform spectral analysis on speech signals to extract the position and time-varying trajectory of formants (i.e., the obvious peaks of speech signals on the spectrum graph), and use linear predictive coding to estimate the position of formants. The frequency of the formants is obtained by solving the roots of the linear predictive coding prediction polynomial.

[0038] Consonant transition acoustic feature extraction: Spectral analysis is performed on the consonant segment to extract the key acoustic features in the consonant transition process. The features are extracted using short-time Fourier transform, which is expressed as:

[0039]

[0040] Where x(n) is the speech signal, w(n) is the window function, N is the number of FFT points, m is the frame index, and k is the frequency index;

[0041] Analysis of phoneme combinations of complex consonants: Combining the formant trajectory and the acoustic characteristics of consonant transitions, the phoneme combinations of complex consonants are analyzed to analyze the pronunciation characteristics, combination methods and relationships of each phoneme in the complex consonants.

[0042] Further, in S3, the specific steps of establishing a single word conversion model and a two-word phrase conversion model from the Amdo dialect to the U-Tsang dialect are:

[0043] Data collection: Collect speech data of the Amdo dialect and the U-Tsang dialect, including single words, two-word words, and sentences, etc. The speech data should contain clear pronunciation and corresponding text annotations;

[0044] Application of the five-degree tone model: The collected Amdo dialect speech data is converted into a five-degree tone representation, which is to more accurately describe the tone information in the dialect. The five-degree tone model is a representation method that divides the tones into five levels according to their height, helping us better understand and process the tone differences in the dialect;

[0045] Data cleaning and annotation: Clean the collected data, remove noise and irrelevant information, and annotate the speech data, including the annotation of tones and vowels;

[0046] LSTM network construction: Use LSTM to build single-word conversion models and double-word word conversion models. The LSTM is a special recurrent neural network that can handle long-term dependencies in sequence data and is very suitable for speech conversion tasks;

[0047] Input and output design: For the single-word conversion model, the input is the single-word speech data of the Amdo dialect (already converted into five-degree tone representation), and the output is the corresponding single-word speech data of the Weizang dialect; for the two-word phrase conversion model, the input is the two-word phrase speech data of the Amdo dialect, and the output is the corresponding two-word phrase speech data of the Weizang dialect.

[0048] Furthermore, in S3, the specific steps of constructing the vowel generation model are:

[0049] Collecting data: collecting speech data of the Amdo dialect and the U-Tsang dialect, wherein the speech data includes samples of long vowels and nasalized vowels;

[0050] Annotate data: Annotate the collected speech data, including parameters such as vowel duration, intensity, nasal resonance, and their changing patterns during the conversion process;

[0051] Select a deep learning framework: Design a vowel generation model based on LSTM.

[0052] Design model structure: The model includes an input layer, a hidden layer and an output layer. The input layer receives the vowel features of the Amdo dialect, the hidden layer is responsible for processing the features and generating the vowel features of the U-Tsang dialect, and the output layer outputs the final U-Tsang dialect vowels;

[0053] Feature extraction: Extract the duration, intensity, and nasal resonance characteristics of vowels from the speech data of the Amdo dialect;

[0054] Feature conversion: According to the characteristics of the Weizang dialect, the extracted features are converted. For example, for long vowels, the duration may need to be adjusted; for nasalized vowels, the intensity of nasal resonance may need to be adjusted;

[0055] Training model: Use the labeled data to train the model so that it can accurately convert the vowels of the Amdo dialect into the vowels of the U-Tsang dialect;

[0056] Optimize the model: Optimize the performance of the model. For example, you can try to use different learning rates, batch sizes and other hyperparameters for training to find the best model configuration.

[0057] Evaluate the model: Use the test data set to evaluate the model and check whether its conversion effect meets the requirements;

[0058] Test the model: Test the performance of the model in actual applications to ensure that it can complete the dialect conversion task stably and accurately.

[0059] Furthermore, in S4, the voice conversion comprises the following specific steps:

[0060] Parameter conversion: Use the single-word conversion model and the two-word word conversion model to convert the extracted Amdo dialect parameters into the Weizang dialect parameters. For long vowels and nasalized vowels, use the vowel generation model to convert them.

[0061] Parameter synthesis: Use the STRAI GHT algorithm to synthesize speech based on the converted Weizang dialect parameters, that is, according to the characteristics of the target speech (such as long vowels and nasal vowels in the Weizang dialect), set the target features, map the source features to the target feature space, adjust the parameters of the speech signal according to the target features, such as duration, sound intensity, spectral characteristics, etc., and use the adjusted parameters to synthesize a new speech signal;

[0062] Synthesis of long vowels and nasalized vowels: When synthesizing new speech signals according to target features, time stretching (in order to generate long vowels, the time length of the source speech signal is stretched, which involves operations such as interpolation or resampling of the signal to increase the time length while maintaining the sound quality), sound intensity adjustment (adjusting the sound intensity according to the target features to ensure that the generated vowels have appropriate loudness and clarity) and spectral feature modification (by adjusting the formant frequency and the degree of nasal resonance, so that the generated vowels have the characteristics of the Weizang dialect, using filters or other signal processing techniques to achieve this);

[0063] Post-processing: Post-process the synthesized speech to improve its naturalness and clarity. Post-processing may include filtering, denoising, volume adjustment, etc.

[0064] Evaluation: The synthesized speech was evaluated using MOS scores and speech quality evaluation indicators. The performance of the conversion model and the effect of the STRAI GHT algorithm were evaluated by comparing the original Weizang dialect speech with the synthesized speech.

[0065] The present invention has the following beneficial effects:

[0066] 1. In the present invention, a special vowel generation model is constructed for the long vowels and nasal vowels in the U-Tsang dialect. The vowel generation model is based on the input Amdo dialect vowels, and generates long vowels and nasal vowels that meet the characteristics of the U-Tsang dialect by adjusting parameters such as the duration, sound intensity and nasal resonance of the vowels. In addition, high-quality long vowels and nasal vowels are generated by synthesis to make up for the shortcomings of the conversion model in generating these phonemes.

[0067] 2. In the present invention, features such as formant trajectory and consonant transition are introduced, and the LSTM model is trained with a large amount of data, so that it can automatically learn and process the complex pronunciation patterns of complex consonants, so as to help the model better capture and distinguish the phoneme combinations of complex consonants. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 The present invention provides a step diagram of a method for converting the speech of the Tibetan Amdo dialect to the U-Tsang dialect. DETAILED DESCRIPTION

[0069] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0070] See also Figure 1 As shown, the present invention is a method for converting the speech of the Tibetan Amdo dialect to the U-Tsang dialect, comprising the following steps:

[0071] S1: Establish corpus: Establish a Tibetan text corpus, and record the parallel speech corpus of the Tibetan Amdo dialect and U-Tsang dialect based on the corpus. Segment, annotate and establish a catalog index for the speech corpus of the Amdo dialect and U-Tsang dialect;

[0072] S2: Acoustic feature analysis and processing: Perform fundamental frequency analysis and duration analysis on single-word and double-word words of Amdo dialect and U-Tsang dialect, compare the differences in fundamental frequency and duration between the two dialects, find out the key parameters of conversion, extract the resonance peak trajectory and consonant transition acoustic features during the analysis, and analyze the phoneme combination of complex consonants. The resonance peak refers to some areas where energy is relatively concentrated in the spectrum of sound. These areas reflect the physical characteristics of the vocal tract (resonance cavity). In the phoneme combination of complex consonants, the resonance peak trajectory of different phonemes can reflect the changes in the shape and size of the vocal tract, which helps to analyze the combination mode and pronunciation characteristics of phonemes. Resonance peak is an important factor in determining sound quality and timbre. In complex consonants, the resonance peaks of different phonemes have different distribution positions and characteristics. Consonant transition acoustic features refer to the acoustic changes produced when consonant phonemes are connected. These changes include changes in frequency, amplitude, timbre, etc. In complex consonants, the transition acoustic features between consonants can reveal the connection relationship and pronunciation characteristics between them, which helps to analyze the structure and pronunciation rules of complex consonants;

[0073] S3: Conversion model establishment: using the five-degree character tone model, combined with LSTM, a single-word conversion model and a two-word word conversion model from the Amdo dialect to the U-Tsang dialect are established. The single-word conversion model and the two-word word conversion model process the generation and conversion of tone information. For the long vowels and nasalized vowels in the U-Tsang dialect, a vowel generation model is constructed. The vowel generation model is based on the input Amdo dialect vowels, and generates long vowels and nasalized vowels that meet the characteristics of the U-Tsang dialect by adjusting the duration, sound intensity and nasal resonance of the vowels;

[0074] S4: Speech conversion: Using the single-word conversion model, the two-word word conversion model and the vowel generation model, the speech conversion from Amdo dialect to U-Tsang dialect is performed through the STRAI GHT algorithm conversion algorithm. In the speech conversion, long vowels and nasalized vowels are generated by synthesis.

[0075] In one embodiment, for the above S1, in S1, the specific steps of establishing the corpus are:

[0076] Build a Tibetan text corpus:

[0077] Collecting text data: Collecting a large amount of Tibetan texts from various Tibetan publications, online resources, academic papers, etc., ensuring that the texts cover a wide range of topics and contexts to reflect the diversity of the language;

[0078] Text preprocessing: clean the collected text, remove irrelevant characters (such as garbled characters, special symbols, etc.), perform word segmentation, split the text into words or phrases, and mark the parts of speech, providing a basis for the subsequent speech corpus construction;

[0079] Original text: "Today is a sunny day, suitable for hiking."

[0080] Text after cleaning: "Today's weather is fine and suitable for hiking"

[0081] Word segmentation result: "Today / the weather / is / clear / suitable / for / going / hiking"

[0082] Parallel speech corpora of Amdo and U-Tsang Tibetan dialects:

[0083] Selection of speakers: Select speakers who are proficient in the Amdo and U-Tsang dialects of Tibetan, ensuring that the speakers are representative and can reflect the characteristics of the dialects;

[0084] Recording speech: Use high-quality recording equipment to record the speaker reading sentences in the text corpus in a quiet environment. Record each sentence multiple times to ensure the clarity and accuracy of the speech.

[0085] Segment, annotate and create a catalog index for the phonetic corpus of the Amdo dialect and the U-Tsang dialect:

[0086] Speech segmentation: Segment the recorded speech into sentences or phrases, and align the speech with the corresponding text using speech recognition software or manual annotation;

[0087] Annotation: Annotate each speech segment, including information such as phonemes, syllables, and tones. Special phonemes such as complex consonants, long vowels, and nasalized vowels require special annotation;

[0088] Create a directory index: Create a unique identifier for each voice segment, create an index table, and associate the identifier with the corresponding text, voice segment, and annotation information;

[0089] The annotation information of the speech segment A1: “[phoneme annotation][syllable annotation][tone annotation]”

[0090] An entry in the index table I: "I(1)=[identifier][text S1][voice PA(S1)][annotation A1]".

[0091] In one embodiment, for the above S2, in S2, the specific steps of the fundamental frequency analysis are:

[0092] Fundamental frequency extraction: The fundamental frequency information of the speech sample is extracted through the autocorrelation function. The fundamental frequency is usually defined as the lowest frequency component of the sound and is the main tone of the sound. The autocorrelation function is expressed as:

[0093]

[0094] Where x(n) is the speech signal, τ is the delay time, and the fundamental frequency corresponds to the delay time that makes the autocorrelation function reach the maximum value;

[0095] Frequency comparison: compare the frequency values ​​of the same single-word and double-word words between the Amdo dialect and the U-Tsang dialect, and analyze the differences in frequency between the two dialects, such as the mean value, standard deviation and other statistics;

[0096] Key parameter extraction: Based on the fundamental frequency comparison results, the key parameters that affect the fundamental frequency difference between the two dialects are extracted. The key parameters include the average value, range and change trend of the fundamental frequency.

[0097] In one embodiment, for the above S2, in S2, the specific steps of the duration analysis are:

[0098] Duration extraction: Extract the duration information of the speech sample. Duration is usually defined as the length of time from the beginning to the end of the speech sample. Duration is expressed as:

[0099] T=t end -t start

[0100] Among them, t start and t end are the start and end time of the speech signal respectively;

[0101] Duration comparison: compare the duration values ​​of the same single-word and double-word words in the Amdo dialect and the U-Tsang dialect, and analyze the differences in duration between the two dialects, such as the mean value, standard deviation and other statistics;

[0102] Key parameter extraction: Based on the duration comparison results, the key parameters that affect the duration difference between the two dialects are extracted. The key parameters include the average value, range and change trend of the duration.

[0103] In one embodiment, for the above S2, in S2, the specific steps of extracting and analyzing the formant trajectory and consonant transition acoustic features;

[0104] Formant trajectory extraction: Perform spectral analysis on speech signals to extract the position and time-varying trajectory of formants (i.e., the obvious peaks of speech signals on the spectrum graph), and use linear predictive coding to estimate the position of formants. The frequency of the formants is obtained by solving the roots of the linear predictive coding prediction polynomial.

[0105] Consonant transition acoustic feature extraction: Spectral analysis is performed on the consonant segment to extract the key acoustic features in the consonant transition process. The features are extracted using short-time Fourier transform, which is expressed as:

[0106]

[0107] Where x(n) is the speech signal, w(n) is the window function, N is the number of FFT points, m is the frame index, and k is the frequency index;

[0108] Analysis of phoneme combinations of complex consonants: Combining the formant trajectory and the acoustic characteristics of consonant transitions, the phoneme combinations of complex consonants are analyzed to analyze the pronunciation characteristics, combination methods and relationships of each phoneme in the complex consonants.

[0109] In one embodiment, for above-mentioned S3, in S3, the specific steps of setting up the single word conversion model and the two-word word conversion model of the Amdo dialect to the U-Tsang dialect are:

[0110] Data collection: Collect speech data of the Amdo dialect and the U-Tsang dialect, including single words, two-word words, and sentences, etc. The speech data should contain clear pronunciation and corresponding text annotations;

[0111] Application of the five-degree tone model: The collected Amdo dialect speech data is converted into a five-degree tone representation, which is to more accurately describe the tone information in the dialect. The five-degree tone model is a representation method that divides the tones into five levels according to their height, helping us better understand and process the tone differences in the dialect;

[0112] Data cleaning and annotation: Clean the collected data, remove noise and irrelevant information, and annotate the speech data, including the annotation of tones and vowels;

[0113] LSTM network construction: Use LSTM to build single-word conversion models and double-word word conversion models. The LSTM is a special recurrent neural network that can handle long-term dependencies in sequence data and is very suitable for speech conversion tasks;

[0114] Input and output design: For the single-word conversion model, the input is the single-word voice data of the Amdo dialect (already converted into five-degree word tone representation), and the output is the corresponding single-word voice data of the Weizang dialect; for the two-word word conversion model, the input is the two-word word voice data of the Amdo dialect, and the output is the corresponding Weizang dialect two-word word voice data;

[0115] Model training: Use the prepared dataset to train the LSTM network;

[0116] Suppose we have a word "a" in the Amdo dialect (the fifth tone is represented by "35"), and we need to convert it to a word in the U-Tsang dialect (The fifth degree is represented by "55"), and this It is a long vowel.

[0117] Input: Voice data of the word "a" (the fifth tone is represented as "35") in the Amdo dialect.

[0118] Processing: First, input the voice data of "a" into the single word conversion model. The model will process its tone information and generate the corresponding single word voice data of the Weizang dialect (but it may not be a long vowel at this time). Then, input the generated voice data into the vowel generation model. The model will adjust the parameters such as the duration, sound intensity and nasal resonance of the vowel according to the input vowel "a" and the characteristics of the Weizang dialect, and generate a long vowel that conforms to the characteristics of the Weizang dialect.

[0119] Output: The final output of the U-Tsang dialect words (the fifth tone is represented by "55"), and this It is a long vowel.

[0120] In one embodiment, for the above S3, in S3, the specific steps of constructing the vowel generation model are:

[0121] Collecting data: collecting speech data of the Amdo dialect and the U-Tsang dialect, wherein the speech data includes samples of long vowels and nasalized vowels;

[0122] Annotate data: Annotate the collected speech data, including parameters such as vowel duration, intensity, nasal resonance, and their changing patterns during the conversion process;

[0123] Select a deep learning framework: Design a vowel generation model based on LSTM.

[0124] Design model structure: The model includes an input layer, a hidden layer and an output layer. The input layer receives the vowel features of the Amdo dialect, the hidden layer is responsible for processing the features and generating the vowel features of the U-Tsang dialect, and the output layer outputs the final U-Tsang dialect vowels;

[0125] Feature extraction: Extract the duration, intensity, and nasal resonance characteristics of vowels from the speech data of the Amdo dialect;

[0126] Feature conversion: According to the characteristics of the Weizang dialect, the extracted features are converted. For example, for long vowels, the duration may need to be adjusted; for nasalized vowels, the intensity of nasal resonance may need to be adjusted;

[0127] Training model: Use the labeled data to train the model so that it can accurately convert the vowels of the Amdo dialect into the vowels of the U-Tsang dialect;

[0128] Optimize the model: Optimize the performance of the model. For example, you can try to use different learning rates, batch sizes and other hyperparameters for training to find the best model configuration.

[0129] Evaluate the model: Use the test data set to evaluate the model and check whether its conversion effect meets the requirements;

[0130] Testing the model: Testing the performance of the model in actual applications to ensure that it can complete the dialect conversion task stably and accurately;

[0131] Suppose we have a vowel "a" in the Amdo dialect, which has a duration of 0.5 seconds, a strength of 80 dB, and no nasal resonance. We want to convert it to a long vowel "ā" in the U-Tsang dialect, which has a duration of 1 second, a strength of 75 dB, and a slight nasal resonance.

[0132] Data preparation: Collect speech data containing "a" and "ā" and annotate them.

[0133] Model design: Design a vowel generation model based on LSTM. The input layer receives the feature vector of "a" and the output layer outputs the feature vector of "ā".

[0134] Feature extraction and conversion: Extract features such as duration and intensity from "a" and convert them into the feature values ​​required for "ā".

[0135] Model training and optimization: Use labeled data to train the model and adjust model parameters to optimize its performance.

[0136] Model evaluation and testing: Use the test dataset to evaluate the performance of the model and test its conversion effect in real applications.

[0137] In one embodiment, for the above S4, in S4, the voice conversion comprises the following specific steps:

[0138] Parameter conversion: Use the single-word conversion model and the two-word word conversion model to convert the extracted Amdo dialect parameters into the Weizang dialect parameters. For long vowels and nasalized vowels, use the vowel generation model to convert them.

[0139] Parameter synthesis: Use the STRAI GHT algorithm to synthesize speech based on the converted Weizang dialect parameters, that is, according to the characteristics of the target speech (such as long vowels and nasal vowels in the Weizang dialect), set the target features, map the source features to the target feature space, adjust the parameters of the speech signal according to the target features, such as duration, sound intensity, spectral characteristics, etc., and use the adjusted parameters to synthesize a new speech signal;

[0140] Synthesis of long vowels and nasalized vowels: When synthesizing new speech signals according to target features, time stretching (in order to generate long vowels, the time length of the source speech signal is stretched, which involves operations such as interpolation or resampling of the signal to increase the time length while maintaining the sound quality), sound intensity adjustment (adjusting the sound intensity according to the target features to ensure that the generated vowels have appropriate loudness and clarity) and spectral feature modification (by adjusting the formant frequency and the degree of nasal resonance, so that the generated vowels have the characteristics of the Weizang dialect, using filters or other signal processing techniques to achieve this);

[0141] Post-processing: Post-process the synthesized speech to improve its naturalness and clarity. Post-processing may include filtering, denoising, volume adjustment, etc.

[0142] Evaluation: The synthesized speech was evaluated using MOS scores and speech quality evaluation indicators. The performance of the conversion model and the effect of the STRAI GHT algorithm were evaluated by comparing the original U-Tsang dialect speech with the synthesized speech.

[0143] Suppose we have a word "da" in the Amdo dialect (corresponding to "dā" in the U-Tsang dialect, which is a long vowel), and we need to convert it to the U-Tsang dialect.

[0144] Parameter extraction:

[0145] Extract the fundamental frequency, duration, spectrum envelope and other parameters of "da".

[0146] Parameter conversion:

[0147] Use the conversion model to convert the fundamental frequency, duration and other parameters of "da" into parameters that conform to the characteristics of "dā".

[0148] In particular, the duration parameter is increased to generate the long vowel "ā".

[0149] Parameter synthesis:

[0150] The STRAI GHT algorithm is used to synthesize the U-Tsang dialect “dā” according to the converted parameters.

[0151] Evaluate:

[0152] Through subjective and objective evaluation, it is verified whether the synthesized "dā" conforms to the characteristics of the U-Tsang dialect, and the performance of the conversion model and the STRAI GHT algorithm is evaluated.

[0153] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for converting the speech of the Tibetan Amdo dialect to the U-Tsang dialect, characterized in that: The following steps are involved: S1: Establish corpus: Establish a Tibetan text corpus, and based on the corpus, record the parallel speech corpus of the Tibetan Amdo dialect and the U-Tsang dialect, segment, annotate and establish a catalog index for the speech corpus of the Amdo dialect and the U-Tsang dialect; S2: Acoustic feature analysis and processing: Conduct fundamental frequency analysis and duration analysis on single-word and double-word words of Amdo dialect and U-Tsang dialect, compare the differences in fundamental frequency and duration between the two dialects, find out the conversion parameters, extract the formant trajectory and consonant transition acoustic features during the analysis, and analyze the phoneme combination of complex consonants; S3: Conversion model establishment: using the five-degree character tone model, combined with LSTM, a single-word conversion model and a two-word word conversion model from the Amdo dialect to the U-Tsang dialect are established. The single-word conversion model and the two-word word conversion model process the generation and conversion of tone information. For the long vowels and nasalized vowels in the U-Tsang dialect, a vowel generation model is constructed. The vowel generation model is based on the input Amdo dialect vowels, and generates long vowels and nasalized vowels that meet the characteristics of the U-Tsang dialect by adjusting the duration, sound intensity and nasal resonance of the vowels; S4: Speech conversion: Using the single-word conversion model, the two-word word conversion model and the vowel generation model, the STRAIGHT algorithm is used to convert the speech from the Amdo dialect to the U-Tsang dialect. During the speech conversion, long vowels and nasalized vowels are generated by synthesis.

2. The method for converting the speech of the Tibetan Amdo dialect to the U-Tsang dialect according to claim 1, characterized in that: In S1, the specific steps of establishing the corpus are: Build a Tibetan text corpus: Collect text data: Collect Tibetan texts from various sources; Text preprocessing: clean the collected text, remove irrelevant characters, perform word segmentation, split the text into words or phrases, and mark the parts of speech; Parallel speech corpora of Amdo and U-Tsang Tibetan dialects: Select speakers: Select speakers who are proficient in the Amdo and U-Tsang dialects of Tibetan; Recording speech: Using a recording device, record the speaker reading sentences in the text corpus; Segment, annotate and create a catalog index for the phonetic corpus of the Amdo dialect and the U-Tsang dialect: Speech segmentation: segment the recorded speech into sentences or phrases, and align the speech with the corresponding text; Annotation: Annotate each speech segment; Create a directory index: Create a unique identifier for each voice segment, create an index table, and associate the identifier with the corresponding text, voice segment, and annotation information.

3. The method for converting the speech of the Tibetan Amdo dialect to the U-Tsang dialect according to claim 1, characterized in that: In S2, the specific steps of the fundamental frequency analysis are: Fundamental frequency extraction: The fundamental frequency information of the speech sample is extracted through the autocorrelation function, which is expressed as: Where x(n) is the speech signal, τ is the delay time, and the fundamental frequency corresponds to the delay time that makes the autocorrelation function reach the maximum value; Frequency comparison: compare the frequency values ​​of the same single-word and double-word words between Amdo dialect and U-Tsang dialect, and analyze the differences in frequency between the two dialects; Key parameter extraction: Based on the fundamental frequency comparison results, the key parameters that affect the fundamental frequency difference between the two dialects are extracted. The key parameters include the average value, range and change trend of the fundamental frequency.

4. The method for converting the speech of the Tibetan Amdo dialect to the U-Tsang dialect according to claim 1, characterized in that: In S2, the specific steps of the duration analysis are: Duration extraction: Extract the duration information of the speech sample. The duration is expressed as: T=t end -t start Among them, t start and t end are the start and end time of the speech signal respectively; Duration comparison: compare the duration values ​​of the same single-word and double-word words in Amdo dialect and U-Tsang dialect, and analyze the differences in duration between the two dialects; Key parameter extraction: Based on the duration comparison results, the key parameters that affect the duration difference between the two dialects are extracted. The key parameters include the average value, range and change trend of the duration.

5. The method for converting the speech of the Tibetan Amdo dialect to the U-Tsang dialect according to claim 1, characterized in that: In S2, the specific steps of extracting and analyzing the formant trajectory and consonant transition acoustic features; Formant trajectory extraction: Perform spectral analysis on speech signals to extract the positions of formants and their changing trajectories over time. Use linear predictive coding to estimate the positions of formants, where the frequencies of formants are obtained by solving the roots of the linear predictive coding prediction polynomial. Consonant transition acoustic feature extraction: Perform spectrum analysis on the consonant segment to extract the key acoustic features in the consonant transition process. Use short-time Fourier transform to extract features. The short-time Fourier transform is expressed as: Where x(n) is the speech signal, w(n) is the window function, N is the number of FFT points, m is the frame index, and k is the frequency index; Analysis of phoneme combinations of complex consonants: Combining the formant trajectory and the acoustic characteristics of consonant transitions, the phoneme combinations of complex consonants are analyzed to analyze the pronunciation characteristics, combination methods and relationships of each phoneme in the complex consonants.

6. The method for converting the speech of the Tibetan Amdo dialect to the U-Tsang dialect according to claim 1, characterized in that: In S3, the specific steps of establishing a single-word conversion model and a two-word word conversion model from the Amdo dialect to the U-Tsang dialect are: Data collection: Collect speech data of Amdo dialect and U-Tsang dialect; Application of the five-degree tone model: convert the collected Amdo dialect speech data into five-degree tone representation; Data cleaning and labeling: Clean the collected data, remove noise and irrelevant information, and label the voice data; LSTM network construction: Use LSTM to build single-word conversion models and double-word word conversion models; Input and output design: For the single-word conversion model, the input is the single-word voice data of the Amdo dialect, and the output is the corresponding single-word voice data of the Weizang dialect; for the two-word word conversion model, the input is the two-word word voice data of the Amdo dialect, and the output is the corresponding Weizang dialect two-word word voice data; Model training: Train the LSTM network using the prepared dataset.

7. The method for converting the speech of the Tibetan Amdo dialect to the U-Tsang dialect according to claim 1, characterized in that: In S3, the specific steps of constructing the vowel generation model are: Collecting data: collecting speech data of the Amdo dialect and the U-Tsang dialect, wherein the speech data includes samples of long vowels and nasalized vowels; Annotated data: annotate the collected speech data; Select a deep learning framework: Design a vowel generation model based on LSTM; Design model structure: The model includes an input layer, a hidden layer and an output layer. The input layer receives the vowel features of the Amdo dialect, the hidden layer is responsible for processing the features and generating the vowel features of the U-Tsang dialect, and the output layer outputs the final U-Tsang dialect vowels; Feature extraction: Extract the duration, intensity, and nasal resonance characteristics of vowels from the speech data of the Amdo dialect; feature Transformation: transform the extracted features; Training model: Use labeled data to train the model; Optimize model: optimize the performance of the model; Evaluate the model: Use the test data set to evaluate the model and check whether its conversion effect meets the requirements.

8. The method for converting the speech of the Tibetan Amdo dialect to the U-Tsang dialect according to claim 1, characterized in that: In S4, the voice conversion comprises the following specific steps: Parameter conversion: Use the single-word conversion model and the two-word word conversion model to convert the extracted Amdo dialect parameters into the Weizang dialect parameters. For long vowels and nasalized vowels, use the vowel generation model to convert them. Parameter synthesis: Use the STRAIGHT algorithm to synthesize speech based on the converted Weizang dialect parameters, that is, according to the characteristics of the target speech, set the target features, map the source features to the target feature space, adjust the parameters of the speech signal according to the target features, and use the adjusted parameters to synthesize a new speech signal; Long vowel and nasal vowel synthesis: When synthesizing new speech signals based on target features, duration stretching, sound intensity adjustment, and spectral feature modification are performed; Post-processing: post-process the synthesized speech; Evaluation: The synthesized speech was evaluated using MOS scores and speech quality evaluation indicators. The performance of the conversion model and the effect of the STRAIGHT algorithm were evaluated by comparing the original U-Tsang dialect speech with the synthesized speech.

Images