A phonetic notation method for the Amdo dialect of Tibetan

By acquiring and processing Tibetan text data, using syllable segmentation model and sequence-to-sequence model, the automatic pronunciation of Tibetan Ando dialects international phonetic symbols is achieved, solving the problems of insufficient data and differences in phonetic mechanisms in the existing technology, and improving the The efficiency and accuracy of phonetic annotation fill the gap in the automatic phonetic annotation of Ando dialect international phonetic symbols.

CN117057322BActive Publication Date: 2024-12-06QINGHAI NORMAL UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311230630.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-09-22
Publication Date
2024-12-06
Estimated Expiration
2043-09-22

AI Technical Summary

Technical Problem

The existing technology has failed to realize the automatic pronunciation of international phonetic symbols in Tibetan Anduo dialects, and the existing research is mainly focused on the Lhasa dialect, which cannot effectively support the automatic pronunciation of international phonetic symbols in Anduo dialects. There are insufficient data, differences in phonetic mechanisms and corpus scale. Limited issues.

Method used

By obtaining Tibetan text data, syllable segmentation is performed based on the Tibetan syllable segmentation model, a standardized syllable knowledge base is constructed, and artificial phonetics and sequence-to-sequence model is combined to establish a base character inverted index to realize the pronunciation of Tibetan and multi-dialects in Tibetan and multi-dialects International phonetic symbols are automatically pronounced.

Benefits of technology

It improves the efficiency and accuracy of the international phonetic annotation of Tibetan Anduo dialects, reduces time and energy investment, fills the gap in the automatic phonetic annotation method of the international phonetic annotation of the international phonetic annotation of the Chinese Anduo dialects, and provides more accurate speech recognition and pronunciation standards.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117057322B_ABST
    Figure CN117057322B_ABST
Patent Text Reader

Abstract

The invention discloses a phonetic notation method for the Tibetan Amdo dialect, which relates to the technical field of natural language processing, and comprises the following steps: obtaining Tibetan text data; performing syllable segmentation on the Tibetan text data based on a Tibetan syllable segmentation model, and constructing a Tibetan standard syllable knowledge base according to the syllable segmentation result; performing manual phonetic notation on the Tibetan standard syllable knowledge base to obtain a Tibetan Amdo dialect standard syllable text pronunciation phonetic notation knowledge base; establishing a base character inverted index according to the Tibetan Amdo dialect standard syllable text pronunciation phonetic notation knowledge base, and performing automatic phonetic notation of the Tibetan Amdo dialect to be phonetized with international phonetic symbols to obtain a phonetic notation result; performing phonetic notation of the Tibetan Amdo dialect with wrong phonetic notation results and not included in the Tibetan Amdo dialect standard syllable text pronunciation phonetic notation knowledge base based on a sequence-to-sequence model. The automatic phonetic notation method of the invention can accurately mark pronunciation features and realize phonetic notation of the Tibetan Amdo dialect.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the technical field of natural language processing, and in particular to a phonetic notation method for the Tibetan Amdo dialect. Background Art

[0002] In the field of linguistics, the International Phonetic Alphabet is widely used to record the characteristics of human speech, and the International Phonetic Alphabet has always been a common method for formally recording the speech of ethnic dialects.

[0003] Regarding the international phonetic transcription of the Amdo dialect of Tibetan, the following bottlenecks need to be solved urgently: (1) The Amdo dialect covers a wide area, and there are significant individual differences among its dialects. Therefore, the international phonetic transcription of the Amdo dialect is an insurmountable basic research for the future automatic phonetic transcription of the Amdo dialect of Tibetan, and it is also the only feasible way at present. Although small-scale data based on dialects can meet the needs of linguistic research, for the speech recognition and text-to-speech conversion research of the Amdo dialect of Tibetan, the speech data based on dialects is insufficient. The principle of standardized syllables and the method based on big data can effectively solve the problem of international phonetic transcription of the Amdo dialect of Tibetan. Therefore, how to determine the written language corpus and expand the scale of the corpus source has become one of the key issues. (2) Up to now, the audio and recording annotation of Tibetan dialects are all completed manually by linguistic professionals after special training. However, the phenomenon of homophones with different annotations is quite frequent, and the process of recording speech is very time-consuming and labor-intensive, and it is difficult to build a large-scale available training data resource. How to use modern information technology methods and knowledge base resource support to realize the automatic international phonetic notation of the pronunciation of the Amdo dialect is extremely important. (3) According to the analysis of existing research results, the research on the international phonetic notation of Tibetan only involves the international phonetic notation of the Lhasa dialect (one of the three major Tibetan dialects), and there is no research result on the automatic phonetic notation of the Amdo dialect of Tibetan. As can be seen from the above, the automatic phonetic notation of the Amdo dialect has important application significance for the speech recognition and synthesis application of the Amdo dialect area in my country.

[0004] So far, there has been no research result on the automatic phonetic notation of the pronunciation of the Amdo Tibetan dialect into the International Phonetic Alphabet. There is only one article on the automatic conversion of the Lhasa Tibetan pronunciation into the International Phonetic Alphabet. This result aims to achieve the automatic conversion of Tibetan text into the Lhasa Tibetan pronunciation into the International Phonetic Alphabet. In the research process, the rule-based method is mainly used to establish the corresponding rule table of consonants, vowels and tones of the Lhasa Tibetan pronunciation, and realize the automatic conversion of the Lhasa Tibetan dialect into the International Phonetic Alphabet. This result has the following shortcomings: (1) From the perspective of phonetics, the Lhasa dialect and the Amdo dialect are quite different, resulting in the automatic phonetic notation of the Lhasa Tibetan dialect and the Amdo dialect into the International Phonetic Alphabet being two research works based on different phonetic notation mechanisms. The research on the automatic phonetic notation of the Lhasa Tibetan dialect into the International Phonetic Alphabet is powerless for the Amdo Tibetan dialect and has little reference value. The differences between the two dialects are mainly reflected in the following aspects: Lhasa dialect has tones, while Amdo dialect has no tones; Lhasa dialect has only complex consonants with pre-nasal consonants, while Amdo dialect has more complex complex consonants; Lhasa dialect has long vowels and nasalized vowels, while Amdo dialect does not; Lhasa dialect has no voiced stops, voiced affricates, and voiced fricative initials, while Amdo dialect has these initials; Amdo dialect has aspirated voiceless fricative initials, while Lhasa dialect does not; Amdo dialect has voiceless nasal initials, while Lhasa dialect basically does not; Lhasa dialect has complex vowels, while Amdo dialect does not; Lhasa dialect has mid-alveolar stops C and Ch, while Amdo dialect has them only in a few areas. (2) "Automatic conversion of Lhasa Tibetan into international phonetic symbols" is to construct the conversion rules of Lhasa sounds and finals based on a limited number of segmented data, and then complete the automatic conversion to international phonetic symbols. The corpus is limited, with only 5,837 sentences of segmented data, and there are annotation errors.

[0005] In summary, the prior art does not have a method for implementing international phonetic notation of the pronunciation of the Tibetan Amdo dialect. Summary of the invention

[0006] The invention provides a phonetic notation method for the Tibetan Amdo dialect, which solves the problem that the prior art lacks a method for automatically notating the pronunciation of the Tibetan Amdo dialect using international phonetic symbols.

[0007] The present invention provides a phonetic notation method for the Tibetan Amdo dialect, comprising the following steps:

[0008] Get Tibetan text data;

[0009] Based on the Tibetan syllable segmentation model, the Tibetan text data is segmented into syllables, and a Tibetan standard syllable knowledge base is constructed according to the syllable segmentation results;

[0010] Manual phonetic notation is performed on the Tibetan standard syllable knowledge base to obtain the Tibetan Amdo dialect standard syllable text pronunciation phonetic notation knowledge base;

[0011] According to the knowledge base of the standard syllables of the Tibetan Amdo dialect, an inverted index of base characters is established, and the Tibetan Amdo dialect to be annotated is automatically annotated with the international phonetic symbols to obtain the phonetic result;

[0012] Based on the sequence-to-sequence model, the pronunciation of the Tibetan Amdo dialect is annotated with international phonetic notation for the incorrect phonetic notation results and the Tibetan Amdo dialect that is not included in the standard syllable pronunciation phonetic notation knowledge base of the Tibetan Amdo dialect.

[0013] Preferably, obtaining Tibetan text data comprises the following steps:

[0014] Collect Tibetan text data resources from the Internet through the Python software Requests library;

[0015] Use crawler technology to obtain a large amount of repeated text in Tibetan text data resources;

[0016] A large amount of duplicate text is deleted through the SimHash algorithm to obtain Tibetan text data.

[0017] Preferably, the Tibetan syllable segmentation model includes an embedding layer, an encoding layer and a decoding layer;

[0018] Syllable segmentation of Tibetan text data is performed based on the Tibetan syllable segmentation model, including the following steps:

[0019] Distributed representation of all characters in Tibetan text data is performed through Word2Vec;

[0020] Extract all character context information in the Tibetan text data after distributed representation through bidirectional LSTM;

[0021] All character context information in the Tibetan text data is decoded through the CRF layer to obtain multiple syllable labels.

[0022] Preferably, the Tibetan text data is segmented into syllables based on the Tibetan syllable segmentation model, and the Tibetan text data needs to be preprocessed, and the preprocessing includes the following steps:

[0023] Tibetan text data is cleaned based on Unicode encoding technology, and a large number of cultural symbols and special character encodings in Unicode encoding are removed;

[0024] The Tibetan sentence segmentation processing technology based on feature templates is used to segment the cleaned Tibetan text data;

[0025] The errors of restoration of connected syllables in Tibetan text data after sentence segmentation are corrected based on the forward maximum matching method, and the spelling errors in Tibetan text data after sentence segmentation are corrected based on the minimum edit distance algorithm.

[0026] Preferably, the Tibetan sentence segmentation processing technology based on the feature template performs sentence segmentation on the cleaned Tibetan text data, comprising the following steps:

[0027] Collect the predicate verbs, predicate adjectives, terminal function words and disjunctive function words of Tibetan syntax to obtain corresponding feature templates;

[0028] Classify sentences with vertical symbols in Tibetan text data based on feature templates;

[0029] The sentence boundaries of the classified Tibetan text data are identified through the support vector machine model.

[0030] Preferably, the correction of the cohesive syllable restoration error of the Tibetan text data after sentence segmentation based on the forward maximum matching method comprises the following steps:

[0031] Perform syllable segmentation on Tibetan text data to obtain multiple syllables;

[0032] Checking the multiple syllables in sequence, if the syllable is not standard, performing a character segmentation operation on the syllable to obtain multiple character segments;

[0033] Multiple characters are segmented into syllables through the maximum matching method, and errors in restoration of joined syllables are corrected.

[0034] Preferably, the spelling errors of the Tibetan text data after sentence segmentation are corrected based on the minimum edit distance algorithm, comprising the following steps:

[0035] Transform one syllable into another syllable using the minimum edit distance algorithm to obtain the minimum number of edit operations;

[0036] The similarity between two syllables is evaluated based on the minimum number of editing operations, and automatic correction of misspelled syllables in Tibetan is achieved.

[0037] Preferably, a base character inverted index is established according to a knowledge base of standard syllable text pronunciation and phonetic notation of the Tibetan Amdo dialect, and the text pronunciation of the Tibetan Amdo dialect to be annotated is automatically annotated with international phonetic symbols to obtain a phonetic notation result, including the following steps:

[0038] Input the phonetic Tibetan Amdo dialect into the base word inverted index to perform base word recognition;

[0039] According to the results of base character recognition, all corresponding Tibetan syllables are screened in the knowledge base, and then matched to obtain the corresponding international phonetic symbols of the Amdo dialect.

[0040] Preferably, the sequence-to-sequence model comprises two RNN networks, one of which serves as an encoder and the other as a decoder;

[0041] Based on the sequence-to-sequence model, the pronunciation of the Tibetan Amdo dialect with incorrect phonetic notation results and not included in the Tibetan Amdo dialect standard syllable pronunciation phonetic notation knowledge base is annotated with international phonetic notation, including the following steps:

[0042] Input the incorrect phonetic notation results and the Tibetan Amdo dialect not included in the Tibetan Amdo dialect standard syllable text pronunciation phonetic notation knowledge base into the encoder, compress it into a vector of a specified length, and obtain a semantic vector;

[0043] The semantic vector is input into the decoder to obtain the output sequence, thus realizing the phonetic notation of the Amdo dialect of Tibetan.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] The present invention first obtains Tibetan text data, then performs syllable segmentation on the Tibetan text data based on a Tibetan syllable segmentation model, and constructs a Tibetan standard syllable knowledge base according to the syllable segmentation result. Performs manual phonetic notation of the Amdo dialect on the Tibetan standard syllable knowledge base to obtain a Tibetan Amdo dialect standard syllable text pronunciation phonetic notation knowledge base. The Tibetan Amdo dialect to be annotated is annotated with the international phonetic notation of the text pronunciation according to the Tibetan Amdo dialect standard syllable text pronunciation phonetic notation knowledge base and a sequence-to-sequence model. The phonetic notation method of the present invention can accurately annotate pronunciation features and realize phonetic notation of the Tibetan Amdo dialect. The phonetic notation efficiency of the Tibetan Amdo dialect international phonetic notation can be effectively improved, and the time and energy investment can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 A flow chart of a Tibetan Amdo dialect phonetic notation method of the present invention;

[0048] Figure 2 This is a structural diagram of the Tibetan syllable segmentation model of the present invention;

[0049] Figure 3 A schematic diagram of encoding a large number of cultural symbols and special characters in the Unicode encoding of the present invention;

[0050] Figure 4 It is a flow chart of the bonding syllable restoration technology of the present invention;

[0051] Figure 5 The present invention is a flow chart of the method for automatic phonetic notation of the international phonetic symbols based on Seq2Seq pronunciation of the Tibetan Amdo dialect. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] Reference Figure 1 The present invention proposes a phonetic notation method for the Tibetan Amdo dialect, which takes the Tibetan natural syllable as the phonetic unit, relies on the big data Tibetan standard syllable recognition technology, the text preprocessing technology for the automatic phonetic notation of the Amdo dialect text pronunciation international phonetic symbols and the Amdo dialect text pronunciation phonetic notation knowledge base, to achieve the automatic phonetic notation of the Amdo dialect text pronunciation international phonetic symbols. It includes the following steps:

[0054] Step 1: Obtain Tibetan text data.

[0055] With the development of the Internet, the scale of Tibetan data resources is growing on the Internet. How to organize these data to build a large-scale Tibetan data resource is an urgent problem to be solved in current resource construction. The present invention uses Python's Requests library to collect and organize a large amount of Tibetan data resources from the Internet. The BeautifulSoup module and regular expressions are combined to solve the HTML web page tag cleaning problem in the data resources, and obtain the main information such as the title, text content, keywords, etc. of the web page. Finally, the title text content is detected by the Tibetan Unicode encoding interval, thereby realizing the automatic detection task of Tibetan web pages.

[0056] There are a lot of duplicate texts in the data resources obtained by crawler technology. Automatically filtering these duplicate texts is an effective way to improve data quality. The present invention uses SimHash algorithm to delete these duplicate text contents. The algorithm is specially used to solve the deduplication task of hundreds of millions of web pages. Its main idea is dimensionality reduction, that is, mapping high-dimensional feature vectors into low-dimensional feature vectors, and determining whether the text is repeated or highly similar by the Hamming distance of the two vectors. The algorithm is divided into 5 steps: text segmentation, calculation of hash value, feature weighting, feature merging, and feature dimensionality reduction. After the text is segmented in the segmentation stage, the feature vector of each word is extracted, and its word frequency is used as the weight. In the hash value calculation stage, the hash value of each word feature vector is calculated by hash function, and the hash value is an n-bit signature composed of binary numbers 01. In the feature weighting stage, all feature vectors are weighted based on the hash value, that is, W=Hash*weight, and when 1 is encountered, the hash value and the weight are positively multiplied, and when 0 is encountered, the hash value and the weight are negatively multiplied. In the feature merging stage, the weighted results of the above feature vectors are accumulated to form a single sequence string. In the feature dimension reduction stage, the accumulated result of the n-bit signature is set to 1 if it is greater than 0, otherwise it is set to 0, so as to obtain the simhash value of the text. Finally, the similarity between texts can be judged based on the Hamming distance of the simhash of different texts. The data filtered by the simhash algorithm basically does not have the problem of text duplication.

[0057] Acquiring Tibetan text data also includes data on traditional Tibetan ancient books and documents.

[0058] Preprocess the Tibetan text data. The preprocessing includes the following steps:

[0059] When the pronunciation of the Tibetan Amdo dialect is automatically annotated with the international phonetic symbol, non-Tibetan characters may have different degrees of negative impact on the annotated results. For this reason, the present invention first cleans the non-Tibetan characters in the text preprocessing stage. The following Tibetan Unicode code range \u0f00-\u0fda is used to limit the character code range, and the characters not in the range are uniformly deleted.

[0060] In addition, the Tibetan Unicode code contains a large number of cultural symbols and special character codes, which have no effect on the automatic phonetic notation of the pronunciation of the Tibetan Amdo dialect. Figure 3 , the present invention eliminates it for clarity.

[0061] The vertical symbol in Tibetan is an important symbol for assisting text recording language. It has the function of indicating the end of a sentence and the function of indicating the pause within a sentence, but it does not have the identification function of the sentence boundary, which makes it more difficult to automatically identify the sentence boundary in the Tibetan text. The present invention starts from the Tibetan syntactic structure, sorts out the real and functional words with the end of sentence identification function such as content predicate verbs and predicate adjectives, function word termination function words and interrogative tone disjunctive function words to design feature templates, as shown in Table 1. 4087 predicate verbs, 148 predicate adjectives, 11 termination function words, and 11 interrogative tone disjunctive function words are collected in the feature template. Finally, on the basis of the feature template, the sentences with vertical symbols in the Tibetan text are classified into sentence boundaries and non-sentence boundaries, and the support vector machine (SVM) model is used to realize the recognition of sentence boundaries. The model linearly classifies the two types of samples in the feature space, and finds the optimal sample in this feature space for linear classification, and its classification decision function is:

[0062] f(x)=sgn(g(x))=sgn(wx+b)

[0063] Where w is the weight vector, x is the feature vector, and b is the parameter. The kernel function is one of the core issues of the support vector machine. The present invention uses the Gaussian kernel function:

[0064]

[0065] Table 1 Tibetan sentence boundary recognition feature template

[0066]

[0067] Through large-scale corpus statistical analysis, it is found that the Tibetan syllables that need to be corrected in the existing corpus are mainly spelling errors and agglutinative syllable restoration errors. Figure 4 The present invention combines the Tibetan standard syllable library and uses the forward maximum matching method to perform the adhesion syllable restoration operation, which solves the problem of syllable loss in real text data. In view of the problem of Tibetan syllable spelling errors, the present invention is based on the Tibetan standard syllable dictionary and uses the minimum edit distance algorithm to convert one syllable into another syllable. The minimum number of edit operations required to evaluate the similarity between two syllables is used to realize the automatic error correction function of Tibetan spelling errors.

[0068] Step 2: Perform syllable segmentation on the preprocessed Tibetan text data based on the Tibetan syllable segmentation model, and build a Tibetan standard syllable knowledge base based on the syllable segmentation results.

[0069] Generally, there are syllable marks between Tibetan syllables for easy identification, but in large-scale Tibetan texts, syllable marks are easily lost, resulting in the adhesion of two or more syllables, which brings certain challenges to the accurate segmentation of Tibetan syllables.

[0070] Reference Figure 2 ,The present paper considers the Tibetan syllable segmentation problem in complex text scenarios, and uses the Bi-LSTM+CRF Tibetan syllable segmentation ,model to solve the Tibetan syllable segmentation task.

[0071] Based on the Tibetan syllable segmentation model, the BIOS data annotation format is used to mark Tibetan characters. Tibetan characters are divided into five categories: placeholder characters and non-placeholder characters, vowel characters, syllables, and punctuation marks, with a total of 16 labels. The Tibetan syllable segmentation model is divided into an embedding layer, an encoding layer, and a decoding layer. The embedding layer uses Word2Vec to perform a distributed representation of all Tibetan characters. In the encoding layer, bidirectional LSTM is used to extract Tibetan character context information and learn the rules of Tibetan characters forming syllables. Finally, the syllable labels of Tibetan characters are jointly decoded by CRF.

[0072] The present invention uses a Tibetan syllable segmentation method based on sequence annotation to realize syllable recognition on large-scale Tibetan pure text, and obtains a total of 9111 standard syllables by counting the syllables, including 8263 pure Tibetan standard syllables and 848 Sanskrit transcription syllables, and constructs a Tibetan standard syllable knowledge base.

[0073] The third step: manually annotate the Tibetan standard syllable knowledge base and the Tibetan Amdo dialect standard syllable knowledge base to obtain the Tibetan Amdo dialect standard syllable text pronunciation and phonetic notation knowledge base.

[0074] In order to improve the accuracy of automatic phonetic notation of the pronunciation of the Tibetan Amdo dialect with international phonetic symbols, the present invention uses Tibetan Amdo dialect dictionaries such as "Amdo Tibetan Spoken Dictionary", "Amdo Dialect Vocabulary", and "Amdo Dialect Conversation Textbook" to construct a corresponding knowledge base of Tibetan Amdo dialect vocabulary and standard syllables.

[0075] Based on the knowledge base of Tibetan standard syllables and the corresponding knowledge base of Tibetan Amdo dialect vocabulary and standard syllables, we have extensively collected many announcers from the mainstream areas of Amdo dialect in Gansu, Qinghai, Sichuan, etc., and carried out the manual phonetic notation of Tibetan language pronunciation in international phonetic symbols to solve the problems of missing, wrong marking and inconsistent phonetic notation in the process of international phonetic notation of Tibetan Amdo dialect text pronunciation. Finally, the construction of the knowledge base of Tibetan Amdo dialect standard syllable text pronunciation notation was realized.

[0076] Step 4: Establish a base character inverted index based on the knowledge base of standard syllables of Amdo Tibetan dialect, and automatically annotate the Amdo Tibetan dialect with international phonetic symbols to obtain the phonetic result. Based on the sequence-to-sequence model, the Amdo Tibetan dialect with wrong phonetic results and not included in the Amdo Tibetan dialect standard syllables of Amdo Tibetan dialect is annotated with international phonetic symbols.

[0077] When realizing the automatic phonetic notation of the international phonetic symbols of the Tibetan Amdo dialect, the present invention adopts a combination of multiple strategies. First, a knowledge base base character inverted index is established for the knowledge base of the Tibetan Amdo dialect standard syllables and the text pronunciation, so as to realize the automatic phonetic notation of the Tibetan Amdo dialect standard syllables to the international phonetic symbols of the text pronunciation. Secondly, for the Tibetan Amdo dialect standard syllables whose phonetic notation results in the knowledge base are inconsistent or not included in the knowledge base, the Seq2Seq method is then used for model training, and the model prediction representation is used as the automatic phonetic notation result to fill in the phonetic notation results in the knowledge base or the international phonetic notation of the text pronunciation of the Tibetan Amdo dialect standard syllables that are not included in the knowledge base.

[0078] In the automatic phonetic notation method based on the knowledge base, the present invention establishes an inverted index of Tibetan base characters on the international phonetic notation library of the pronunciation of the Tibetan Amdo dialect. This technology matches the Tibetan syllables to be annotated according to the Tibetan syllable base character screening results, and returns the international phonetic notation of the pronunciation of the Tibetan Amdo dialect, which effectively improves the phonetic notation efficiency of the international phonetic notation method of the Tibetan Amdo dialect pronunciation when facing large-scale Tibetan texts.

[0079] In the automatic phonetic notation method based on Seq2Seq, the present invention utilizes two RNNs, one of which is used as an encoder and the other as a decoder. The encoder is responsible for compressing the input Tibetan text sequence into a vector of specified length, namely the semantic vector C. This vector can be regarded as a transformation of all implicit states of the input Tibetan text sequence to obtain the semantic vector. The decoder is responsible for generating an international phonetic notation sequence of the pronunciation of the Tibetan Amdo dialect based on the semantic vector. In this step, the semantic vector C is input into the RNN of the encoder as the initial state to obtain the output sequence. At this time, the output of the previous moment will become the input of the current moment, and the semantic vector C will participate in the calculation of all moments of the sequence, and finally output the international phonetic notation sequence of the pronunciation of the Tibetan Amdo dialect. The specific model structure is as follows: Figure 5 shown.

[0080] The invention of the automatic phonetic notation method of the international phonetic symbol of the text pronunciation of the Tibetan Amdo dialect can solve a series of problems such as the phonetic notation error, low efficiency phonetic notation, non-standard phonetic notation, and narrow dialect coverage of the manual annotation of the international phonetic symbol of the Amdo dialect, making the pronunciation and notation of the Tibetan Amdo dialect more accurate, fast and automatic, and can fill the gap of the lack of automatic phonetic notation method of the international phonetic symbol in the field of the Tibetan Amdo dialect. At present, no automatic phonetic notation system for the international phonetic symbol of the Tibetan dialect has been developed at home and abroad, and the only manual phonetic notation method has multiple defects. Specifically: due to individual differences in the recorders, the manual phonetic notation results of the Tibetan Amdo dialect often have errors. The automatic phonetic notation method can accurately mark the pronunciation characteristics and improve the accuracy of phonetic notation; the manual phonetic notation lacks standardization, and the present invention adopts the text pronunciation and the pronunciation of the announcer, and the phonetic notation is more standardized; the manual phonetic notation has the defect of a narrow coverage area, and the present invention covers the mainstream Tibetan Amdo dialect areas such as Gansu, Qinghai, and Sichuan; the method based on manual phonetic notation is time-consuming and laborious, and the present invention can effectively improve the phonetic notation efficiency of the international phonetic symbol of the Tibetan Amdo dialect, reducing time and energy investment.

[0081] The international phonetic notation method for the pronunciation of the Tibetan Amdo dialect has the following beneficial effects and economic benefits:

[0082] It helps to accurately pronounce the pronunciation of the Tibetan Amdo dialect: The international phonetic notation method for the pronunciation of the Tibetan Amdo dialect can help people who study and research the Amdo dialect to pronounce Tibetan syllables more accurately, avoid inaccurate pronunciation problems, and improve language acquisition efficiency by accurately noting the pronunciation position and pronunciation method of each syllable.

[0083] Improve speech recognition accuracy: The Amdo dialect is one of the three major Tibetan dialects, and is unique in its word usage and pronunciation features compared to other dialects. The Amdo pronunciation international phonetic notation method can provide more accurate speech recognition results and improve accuracy.

[0084] Promote the standardization of Tibetan speech technology: The popularization of the international phonetic notation method of the Amdo dialect can promote the standardization of Tibetan speech technology and improve the interoperability and operability of Tibetan speech technology.

[0085] Promote the development of Tibetan speech technology: The popularization of the international phonetic notation system for the Amdo dialect can promote the development of Tibetan speech technology, thereby providing more accurate and reliable basic data for the research and application of Tibetan speech technology.

[0086] Economic benefits: The popularization of the international phonetic notation method of Ando pronunciation can bring economic benefits. For example, it can reduce costs and improve efficiency in application fields such as Tibetan speech recognition, Tibetan speech synthesis, and Tibetan speech translation.

[0087] In short, the emergence of the international phonetic notation method of Amdo pronunciation is of great significance to the research and application of Tibetan speech technology. It can provide more accurate and reliable basic data for the research and application of Tibetan speech technology, thereby promoting the development and application of Tibetan speech technology.

[0088] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0089] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A phonetic notation method for the Tibetan Amdo dialect, characterized in that: The following steps are involved: Get Tibetan text data; Based on the Tibetan syllable segmentation model, the Tibetan text data is segmented into syllables, and a Tibetan standard syllable knowledge base is constructed according to the syllable segmentation results; Performing manual phonetic notation on the Tibetan standard syllable knowledge base to obtain the Tibetan Amdo dialect standard syllable text pronunciation phonetic notation knowledge base; According to the knowledge base of the standard syllables of the Tibetan Amdo dialect, an inverted index of base characters is established, and the Tibetan Amdo dialect to be annotated is automatically annotated with the international phonetic symbols to obtain the phonetic result; Based on the sequence-to-sequence model, the pronunciation of the Tibetan Amdo dialect that has errors in the phonetic notation results and is not included in the knowledge base of the standard syllables of the Tibetan Amdo dialect is annotated with international phonetic notation. Before syllable segmentation is performed on the Tibetan text data based on the Tibetan syllable segmentation model, the Tibetan text data needs to be preprocessed, and the preprocessing includes the following steps: Tibetan text data is cleaned based on Unicode encoding technology, and a large number of cultural symbols and special character encodings in Unicode encoding are removed; The Tibetan sentence segmentation processing technology based on feature templates is used to segment the cleaned Tibetan text data; Based on the forward maximum matching method, the errors of restoration of the attached syllables in the Tibetan text data after segmentation are corrected, and based on the minimum edit distance algorithm, the spelling errors in the Tibetan text data after segmentation are corrected; The Tibetan sentence segmentation processing technology based on feature templates performs sentence segmentation on the cleaned Tibetan text data, including the following steps: Collect the predicate verbs, predicate adjectives, terminal function words and disjunctive function words of Tibetan syntax to obtain corresponding feature templates; Classify sentences with vertical symbols in Tibetan text data based on feature templates; Sentence boundary recognition is performed on the classified Tibetan text data through support vector machine model; The sequence-to-sequence model includes two RNN networks, one of which serves as an encoder and the other as a decoder; Based on the sequence-to-sequence model, the pronunciation of the Tibetan Amdo dialect with incorrect phonetic notation results and not included in the Tibetan Amdo dialect standard syllable pronunciation phonetic notation knowledge base is annotated with international phonetic notation, including the following steps: Input the incorrect phonetic notation results and the Tibetan Amdo dialect not included in the Tibetan Amdo dialect standard syllable text pronunciation phonetic notation knowledge base into the encoder, compress it into a vector of a specified length, and obtain a semantic vector; The semantic vector is input into the decoder to obtain the output sequence, thus realizing the phonetic notation of the Amdo dialect of Tibetan.

2. A method for phonetic notation of the Tibetan Amdo dialect as claimed in claim 1, characterized in that: Obtaining Tibetan text data includes the following steps: Collect Tibetan text data resources from the Internet through the Python software Requests library; Use crawler technology to obtain a large amount of repeated text in Tibetan text data resources; A large amount of duplicate text is deleted through the SimHash algorithm to obtain Tibetan text data.

3. A method for phonetic notation of the Tibetan Amdo dialect as claimed in claim 1, characterized in that: The Tibetan syllable segmentation model includes an embedding layer, an encoding layer and a decoding layer; Syllable segmentation of Tibetan text data is performed based on the Tibetan syllable segmentation model, including the following steps: Distributed representation of all characters in Tibetan text data is performed through Word2Vec; Extract all character context information in the Tibetan text data after distributed representation through bidirectional LSTM; All character context information in the Tibetan text data is decoded through the CRF layer to obtain multiple syllable labels.

4. A method for phonetic notation of the Tibetan Amdo dialect as claimed in claim 1, characterized in that: The method of correcting the restoration error of the connected syllables of the Tibetan text data after the sentence segmentation based on the forward maximum matching method includes the following steps: Perform syllable segmentation on Tibetan text data to obtain multiple syllables; Checking the multiple syllables in sequence, if the syllable is not standard, performing a character segmentation operation on the syllable to obtain multiple character segments; Multiple characters are segmented into syllables through the maximum matching method, and errors in restoration of joined syllables are corrected.

5. A method for phonetic notation of the Tibetan Amdo dialect as claimed in claim 4, characterized in that: Correcting spelling errors of Tibetan text data after segmentation based on the minimum edit distance algorithm includes the following steps: Transform one syllable into another syllable using the minimum edit distance algorithm to obtain the minimum number of edit operations; The similarity between two syllables is evaluated based on the minimum number of editing operations, and automatic correction of misspelled syllables in Tibetan is achieved.

6. A method for phonetic notation of the Tibetan Amdo dialect as claimed in claim 1, characterized in that: The method includes the following steps: establishing a base character inverted index according to a knowledge base of standard syllables of the Tibetan Amdo dialect, automatically annotating the pronunciation of the Tibetan Amdo dialect with international phonetic symbols, and obtaining a phonetic result, including the following steps: Input the phonetic Tibetan Amdo dialect into the base word inverted index to perform base word recognition; According to the results of base character recognition, all corresponding Tibetan syllables are screened in the knowledge base, and then matched to obtain the corresponding international phonetic symbols of the Amdo dialect.