Tibetan dialect language conversion method

Abstract

The invention discloses a rule-based Tibetan dialect language conversion method. The method comprises the following steps: S1, constructing a dialect and virtual word knowledge base, spoken words and a virtual word rule base by using dialect data and Tibetan grammar; s2, establishing a conversion model by using the constructed virtual word rule base; s3, collecting data by using the conversion model, and carrying out data preprocessing; and S4, based on the processed data, using a conversion model to carry out dialect language recognition, and then outputting a written language. Compared with the prior art, the rule-based Tibetan dialect language conversion method has the advantages that the detailed Tibetan language and virtual word rule base is constructed, so that the conversion model is formed.

Description

Technical Field

[0001] The present invention relates to the technical field of Tibetan speech and text data processing, and in particular to a Tibetan dialect style conversion method. Background Art

[0002] The diversity of Tibetan dialects poses significant challenges to fields such as speech recognition, machine translation, and speech data augmentation.

[0003] In terms of vocabulary, pronunciation, grammar and pragmatics, there are significant differences between the three major Tibetan dialects and the written language. These differences are mainly reflected in the use of function words. Function words are divided into unrestricted function words and free function words. The difference lies in whether they are restricted by the addition of a character after the previous syllable or by the addition of another character after the previous syllable. Dialect function words refer to words in the three major dialects that have the same semantics as the function words in the written language. They differ from the function words in the written language in form and usage. Although the dialects also include terminal words, words to be described, suffixes referring to personal nouns and other function words (such as " (zhing)”), but the usage of these function words has not changed significantly.

[0004] Among them, la case function words include (su), (ra), (ru), (du), (na), (la), (tu), which is almost replaced by other variants of function words in the three major Tibetan dialects, and their addition rules are also different:

[0005] In the U-Tsang dialect, (ru) In addition to adding a word at the end (va) or without additional characters, and is often used after other additional characters;

[0006] In the Ando dialect, the choice of function words is based on the characters added after the previous syllable, and these added characters are usually used directly as function words.

[0007] As for the Kham dialect, it tends to use a variety of (la) sound variants to connect sentences. For example: Kham dialect (Recorded in ancient books.), U-Tsang dialect (Going to work.) and the Amdo dialect (Enters the house.) Wait.

[0008] As a result, the research on stylistic transformation of Tibetan dialect is not comprehensive and lacks a certain degree of systematicity. Summary of the Invention

[0009] The technical problem to be solved by the present invention is to overcome the above technical defects and provide a Tibetan dialect style conversion method that constructs addition rules through the non-free function words of the three major dialects to perform style rule conversion.

[0010] To solve the above technical problems, the present invention provides a technical solution: a Tibetan dialect body conversion method, comprising the following steps:

[0011] S1: Use dialect data and Tibetan grammar to build a dialect function word knowledge base, colloquial words, and function word rule base;

[0012] S2: Establish a conversion model using the constructed function word rule base;

[0013] S3: Use the conversion model to collect data and perform data preprocessing;

[0014] S4: Based on the processed data, the conversion model is used to perform dialect style recognition and then output the written language.

[0015] Preferably, the dialect data includes Amdo dialect, U-Tsang dialect, and Kham dialect.

[0016] Preferably, the function word features of the function word rule base in S1 are constructed based on the non-free function words of la case function words, nominative case function words, genitive case function words, decorative function words and separable function words;

[0017] Specifically divided into: post-added characters, further post-added characters, dialect function words, written language function words and la case function words;

[0018] After symbol mapping, a formal description is performed, and based on the formal description, rules for adding written function words and conversion rules for dialect function words are constructed.

[0019] Preferably, the non-self-owned function words of the la form function words include (su), (ra), (ru), (tu), (na), (la), (du).

[0020] Preferably, the symbol mapping is:

[0021] Indicator a1~a 11 、b1~b i 、v1~v 29 ;

[0022] Among them, a1~a 11 Mapping post-addition characters and no post-addition characters, b1~b i Mapping dialect function words, v1~v 29Mapping written language function words;

[0023] The formal description includes:

[0024] Set Suffix={a1....a 10} respectively correspond to 10 Tibetan characters followed by additional characters;

[0025] gather Then any a i ∈Postfix, i=3,10 corresponding Tibetan characters are added after it;

[0026] Set Localism = {b1, b2, b3, b4, b5, b6, b7, ... b 32 , b 33 , b 34} elements correspond to 34 dialect function words;

[0027] Set function = {v1, v2, v3, v4, v5, ... v 28 , v 29} respectively correspond to 29 function words in written language;

[0028] gather Corresponding to the Tibetan la-case function word.

[0029] Preferably, the written language function word addition rules include:

[0030] a i ∈suffix, i=1,5 corresponding to the suffix and a i ∈Postfix, i=3 corresponding to the next word can only be added to v1∈Lage;

[0031] a i ∈suffix, i=2, 3, 4, 6, 8, 9 corresponding to the suffixed character, the syllable after can only be added to v2∈Lage;

[0032] a i ∈suffix, i=10 corresponding to the suffix and a i ∈Postfix, i=10 The corresponding subsequent characters can only be added to v3∈Lage;

[0033] a i ∈suffix, the corresponding suffix character of i=7 can only be appended to v4 or v5∈Lage.

[0034] Preferably, the dialect function word conversion rules include:

[0035] gather Then any dialect function word bi, i=2, 8, 15, 14, appears in ai ∈suffix,i=1,5 or a i ∈Postfix, when i=3, it should be replaced by v1∈Lage;

[0036] For collections Then any dialect function word b i Appears in a 10 ∈suffi, it should be replaced by v3∈Lage.

[0037] For collections Then any dialect function word b i Appears in a i ∈suffix, i=2, 3, 4, 6, 8, should be replaced by v2∈Lage;

[0038] For collections Then any dialect function word b i When it appears after a7∈suffix, it should be replaced by v4∈Lage.

[0039] Preferably, the data collection includes dialect text data and dialect voice data. When the collected data is dialect voice data, data preprocessing is performed after voice recognition.

[0040] The data preprocessing includes text cleaning, part-of-speech tagging, and manual proofreading.

[0041] The advantages of the present invention over the prior art are: based on the existing spoken dictionaries of the three major Tibetan dialects, the spoken words of the Tibetan dialect content words and the corresponding written content words are constructed through screening, and at the same time a dialect function word knowledge base is constructed. Through the la case function words, nominative case function words, genitive case function words, decorative function words and non-free function words in the three major dialects, a detailed Tibetan function word rule base is constructed, thereby forming a conversion model, completing the Tibetan dialect style conversion, and providing a high-accuracy conversion method that is easy to promote and apply. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a schematic diagram of the framework structure of a Tibetan dialect style conversion method. DETAILED DESCRIPTION

[0043] Next pair combination Figure 1 The present invention is described in further detail.

[0044] Those skilled in the art should understand that the present invention is not limited to the following embodiments. The following embodiments and descriptions are only for illustrative purposes of the present invention. Without departing from the spirit and scope of the present invention, the present invention may be subject to various changes and improvements, which shall fall within the scope of the invention to be protected.

[0045] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative work shall fall within the scope of protection of the present invention.

[0046] A Tibetan dialect style conversion method comprises S1: using dialect data and Tibetan grammar to construct a dialect function word knowledge base, a spoken word and function word rule base; S2: using the constructed function word rule base to establish a conversion model; S3: using the conversion model to collect data and perform data preprocessing; S4: using the conversion model to perform dialect style recognition based on the processed data and then output written language.

[0047] When the present invention is specifically implemented:

[0048] Definition of terms:

[0049] Add words afterward:

[0050] In Tibetan, specific Tibetan consonant letters that appear after a base character or a vertical combination character and are spelled with the base character are called suffixed characters of Tibetan.

[0051] Add words at the end:

[0052] In Tibetan, specific Tibetan consonant letters that appear after the suffix and spelled with the base letter are called suffix characters of Tibetan.

[0053] Dialect function words:

[0054] The characters or words that play the role of Tibetan function words in sentences of the three major Tibetan dialects are called dialect function words.

[0055] Function words in written language:

[0056] In a sentence, a word cannot stand alone as a grammatical component or the main bearer of grammatical meaning, but can only be attached to a content word or a sentence to express grammatical meaning or additional meaning and play a certain grammatical role.

[0057] La case function words:

[0058] The la-case particle has a variety of uses. It is used after static nouns, pronouns, and noun phrases to form a la-case particle structure. Depending on the relationship between the following predicate verb and the different properties, it can serve as the object, adverbial, complement, or even the subject in the sentence. It is used after the verb to act as a conjunction to connect clauses.

[0059] This paper focuses on the la-case function words, nominative function words, genitive function words, decorative function words, and non-free function words in the three major dialects, and constructs a detailed Tibetan function word rule library. The Tibetan function word addition rules show a relatively fixed connection relationship due to the strict restrictions of the post-added characters. In order to ensure the completeness and practicality of the rule library, ten post-added characters were considered, and the usage of dialect function words was statistically analyzed: Table 1 Characteristics and examples of la-case function words in dialects

[0060]

[0061] Note: The “” mark represents a dialect function word. “()” represents a written function word.

[0062] Symbol Mapping:

[0063] In order to formally describe the complex function words in Tibetan dialects, the 10 suffixed characters and no suffixed characters are mapped to a1~a 11 Indicator, some dialect function words are mapped to b1~b i Indicator, and 29 written function words are mapped to v1~v 29 Indicator. Details are shown in Table 1-5:

[0064] Table 2 Mapping relationship between suffix letters and letter representations

[0065]

[0066] Note: “—” means no suffix.

[0067] Table 3 Mapping relationship between dialect function words and letter representations

[0068]

[0069] Note: "…" and "bi-1, bi" indicate that in addition to the listed dialect function words, there may be other dialect function words;

[0070] The diversity and complexity of dialects means that further additions and refinement of these function words may be necessary in the future.

[0071] Table 4 Mapping relationship between written function words and letter representations

[0072]

[0073] Table 11a form function word rule library reference table

[0074]

[0075]

[0076] Based on the above content, the following Tibetan dialect function words are formally described using set theory:

[0077] ①Set Suffix={a1....a 10}'s elements correspond to 10 Tibetan characters followed by additional characters;

[0078] ②For the set Then any a i ∈Postfix, i=3,10 corresponding Tibetan characters are added at the end;

[0079] ③ Set Localism={b1,b2,b3,b4,b5,b6,b7,……b 32 ,b 33 ,b 34} elements correspond to 34 dialect function words;

[0080] ④ Set function = {v1,v2,v3,v4,v5,...v 28 ,v 29}'s elements correspond to 29 written function words;

[0081] ⑤For the collection The corresponding Tibetan la-case function word;

[0082] Rules for adding function words in written language:

[0083] ①For the collection Then any dialect function word b i (where i=2,8,15,14,4) appears in a i ∈suffix,i=1,5 or a i ∈ Postfix, i=3, it should be replaced by v1∈Lage.

[0084] ②For the set Then any dialect function word b i (where the value of i is defined in the set) appears in a 10 ∈suffix, it should be replaced by v3∈Lage.

[0085] ③For the collection Then any dialect function word b i (where the value of i is defined in the set) appears in a i ∈suffix,i=2,3,4,6,8,9, it should be replaced by v2∈Lage.

[0086] ④For the collection Then any dialect function word b i(where the value of i is defined in the set) should be replaced by v4∈Lage when it appears after a7∈suffix.

[0087] When the function word rule base and knowledge base constructed above are used in Tibetan language conversion:

[0088]

[0089]

[0090] First, extract the part of speech of all syllables in the line, count the number of parts of speech that meet the conversion requirements, and obtain the position of the part of speech in the current line. If the suffix of the syllable that meets the conversion requirements is in the knowledge base of dialect function words and the next syllable is the corresponding dialect function word, then replace the current syllable with the corresponding written language function word R and update the converted text. For example: dialect sentence The result of the voice conversion of (How to do this job?) is (How does this job work?).

[0091] Comparative experiment:

[0092] The data used in the present invention comes from self-built dialect text data and text data transcribed by a Tibetan dialect speech recognition model.

[0093] The self-built dialect text data comes from dialect books and statistical data on the Tibetan dialect spatiotemporal data service platform. The dataset covers daily sentences in the three major dialects.

[0094] During the data preprocessing phase, the present invention first proofread the text data transcribed by the Tibetan dialect speech recognition model. Next, part-of-speech tagging was performed using the TIP-LAS open-source tool. Further proofreading was performed, and the data for the three major dialects was divided according to dialect categories. The data volumes for the three major dialects are shown in Table 6.

[0095] Table 6 Data volume of the three major dialects

[0096]

[0097] The accuracy (A) is used as the evaluation index, and the specific formula is as follows:

[0098]

[0099] Here, a represents the number of correct sentences after the voice conversion, and e represents the number of sentences that need to be converted into a new voice.

[0100] result:

[0101] Table 2 Experimental results

[0102]

[0103] As shown in Table 7 above, the average accuracy of the Tibetan function word style conversion method proposed in this study reached 72.68%, verifying the effectiveness of the method proposed in this invention.

[0104] Table 3 Examples of incorrect language conversion

[0105]

[0106]

[0107] The contents not described in detail in this specification belong to the prior art known to those skilled in the art.

[0108] The present invention and its implementation methods are described above. This description is not restrictive. If ordinary technicians in this field are inspired by it and design structural methods and embodiments similar to the technical solution without creatively designing them without departing from the purpose of the invention, they should all fall within the scope of protection of the present invention.

Claims

1. A Tibetan dialect body language conversion method, characterized by: The following steps are involved: S1: Use dialect data and Tibetan grammar to build a dialect function word knowledge base, colloquial words, and function word rule base; S2: Establish a conversion model using the constructed function word rule base; S3: Use the conversion model to collect data and perform data preprocessing; S4: Based on the processed data, the conversion model is used to perform dialect style recognition and then output the written language.

2. The method for converting Tibetan dialect into a dialect according to claim 1, wherein: The dialect data includes Amdo dialect, U-Tsang dialect, and Kham dialect.

3. The method for converting Tibetan dialect into a dialect according to claim 1, wherein: The function word features of the function word rule base in S1 are constructed based on the non-free function words of la case function words, nominative function words, genitive function words, decorative function words and separable function words; Specifically divided into: post-added characters, further post-added characters, dialect function words, written language function words and la case function words; After symbol mapping, a formal description is performed, and based on the formal description, rules for adding written function words and conversion rules for dialect function words are constructed.

4. The method for converting Tibetan dialect into a dialect according to claim 3, wherein: The non-self-owned function words of the la-case function words include (su), (ra), (ru), (du), (na), (la), (tu).

5. The method for converting Tibetan dialect into a dialect according to claim 3, wherein: The symbol mapping is: Indicator a1~a 11 、b1~b i 、v1~v 29 ; Among them, a1~a 11 Mapping post-addition characters and no post-addition characters, b1~b i Mapping dialect function words, v1~v 29 Mapping written language function words; The formal description includes: Set Suffix={a1....a 10 } respectively correspond to 10 Tibetan characters followed by additional characters; gather Then any a i ∈Postfix, i=3,10 corresponding Tibetan characters are added after it; Set Localism = {b1, b2, b3, b4, b5, b6, b7, ... b 32 , b 33 , b 34 } elements correspond to 34 dialect function words; Set function = {v1, v2, v3, v4, v5, ... v 28 , v 29 } respectively correspond to 29 function words in written language; gather Corresponding to the Tibetan la-case function word.

6. The method for converting Tibetan dialect into a dialect according to claim 5, wherein: The written language function word addition rules include: a i ∈suffix, i=1,5 corresponding to the suffix and a i ∈Postfix, i=3 corresponding to the next word can only be added to v1∈Lage; a i ∈suffix, i=2, 3, 4, 6, 8, 9 corresponding to the suffixed character, the syllable after can only be added to v2∈Lage; a i ∈suffix, i=10 corresponding to the suffix and a i ∈Postfix, i=10 The corresponding subsequent characters can only be added to v3∈Lage; a i ∈suffix, the corresponding suffix character of i=7 can only be appended to v4 or v5∈Lage.

7. The method for converting Tibetan dialect into a dialect according to claim 5, wherein: The dialect function word conversion rules include: gather Then any dialect function word bi, i=2, 8, 15, 14, appears in a i ∈suffix,i=1,5 or a i ∈Postfix, when i=3, it should be replaced by v1∈Lage; For collections Then any dialect function word b i Appears in a 10 ∈suffi, it should be replaced by v3∈Lage. For collections Then any dialect function word b i Appears in a i ∈suffix, i=2, 3, 4, 6, 8, should be replaced by v2∈Lage; For collections Then any dialect function word b i When it appears after a7∈suffix, it should be replaced by v4∈Lage.

8. The method for converting Tibetan dialect into a dialect according to claim 1, wherein: The data collection includes dialect text data and dialect voice data. When the collected data is dialect voice data, data preprocessing is performed after voice recognition; The data preprocessing includes text cleaning, part-of-speech tagging, and manual proofreading.

Images