Automatic construction method of myanmar-english parallel corpus in large model based on multi-dimensional evaluation

Abstract

This invention relates to a method for automatically constructing a large-scale Chinese-Myanmar parallel corpus based on multidimensional evaluation, belonging to the field of natural language processing technology. To address the scarcity of Chinese-Myanmar parallel corpus resources, this invention proposes a method for automatically constructing a large-scale Chinese-Myanmar parallel corpus based on multidimensional evaluation. First, Burmese corpora are manually collected to construct a basic dataset. Second, prompt words are designed to generate a candidate set of pseudo-parallel Chinese-Burmese corpora using a large-scale model. Then, the candidate set is filtered using semantic similarity constraints. Finally, the large-scale model is used to score the filtered candidate set from multiple dimensions, and the corpora are ranked and hierarchically filtered based on the scores to obtain high-quality pseudo-parallel corpora. Experimental results show that the corpus constructed by this invention can effectively improve the model's translation performance and significantly enhance its translation quality in low-resource Chinese-Myanmar language environments.

Description

Technical Field

[0001] This invention relates to a method for automatically constructing a large-scale parallel Chinese-Myanmar corpus based on multidimensional evaluation, belonging to the field of natural language processing technology. Background Technology

[0002] In the field of natural language processing, parallel corpora are crucial resources for training machine translation models. Especially for low-resource language pairs (such as Chinese-Myanmar language pairs), the lack of parallel corpora becomes a bottleneck restricting the improvement of translation quality. Currently, most Chinese-Myanmar parallel corpora come from human translations or existing publicly available corpora, which are often limited in quantity and inconsistent in quality. Therefore, how to efficiently and automatically generate high-quality Chinese-Myanmar parallel corpora has become a pressing problem in the field of machine translation.

[0003] Existing methods utilize traditional data cleaning and augmentation techniques, but these often rely on human intervention and struggle to guarantee sufficient translation quality in low-resource language environments. Meanwhile, the development of large-scale models (such as the GPT series and Llama) in recent years has offered new possibilities for generating high-quality pseudo-parallel corpora, particularly in machine translation tasks for low-resource languages, effectively addressing the problem of low translation quality. However, how to combine large-scale model generation and evaluation mechanisms to ensure the quality of the final generated parallel corpora remains a challenge that has not yet been fully resolved. Summary of the Invention

[0004] This invention proposes an automatic method for constructing large-scale Chinese-Myanmar parallel corpora based on multidimensional evaluation. The aim is to significantly improve machine translation performance on low-resource language pairs by automatically constructing high-quality Chinese-Myanmar parallel corpora. This method uses a multidimensional evaluation mechanism to filter generated pseudo-parallel corpora, thereby improving data quality and ultimately providing high-quality training data for machine translation models.

[0005] The technical solution of this invention is: an automatic construction method for a large-scale Chinese-Myanmar parallel corpus based on multidimensional evaluation. The specific steps of the automatic construction method for a large-scale Chinese-Myanmar parallel corpus based on multidimensional evaluation are as follows:

[0006] Step 1: Prepare candidate datasets; collect parallel English-Myanmar bilingual corpora, and construct parallel candidate datasets for English-Myanmar-Chinese trilingual corpora through large-scale model translation.

[0007] Step 2: Calculate semantic similarity to filter candidate sets and select English-Chinese-Myanmar parallel corpus datasets;

[0008] Step 3: Construct large-scale model prompt words and score the trilingual parallel corpus using the large-scale model;

[0009] Step 4: Rank the corpora according to the comprehensive scores, select high-quality data to construct a high-quality parallel corpus of Chinese and Burmese.

[0010] Furthermore, the specific steps of Step 1 are as follows:

[0011] Step 1.1: Collect Burmese-English parallel corpora from the Hugging Face website to create candidate datasets;

[0012] Step 1.2: Use the LLaMA 3.1-8b model to translate the collected English-Myanmar parallel corpus and construct a parallel candidate dataset of English, Burmese and Chinese.

[0013] Step 1.3: After initial data cleaning, noisy data is removed to ensure the quality of the candidate dataset.

[0014] Furthermore, the specific steps of Step 2 are as follows:

[0015] Step 2.1: Calculate the semantic similarity between each team of candidate data using cosine similarity.

[0016] Step 2.2: Using a weighted average strategy, the similarity of each pair of candidate corpora is calculated by weighting, and high-quality English-Chinese-Myanmar parallel corpus datasets are selected based on the comprehensive semantic similarity value.

[0017] Furthermore, the specific steps of Step 3 are as follows:

[0018] Step 3.1: Design a simple and efficient input template for a large language model system based on task orientation, by providing a clear structure and guiding prompts;

[0019] Step 3.2: Use the LLaMA 3.1-8b model to score the trilingual parallel corpus. The scoring criteria include translation accuracy, grammatical structure, and language fluency.

[0020] The present invention also provides an automatic construction system for a large-scale Chinese-Myanmar parallel corpus based on multidimensional evaluation, the system comprising: a module for executing the automatic construction method for a large-scale Chinese-Myanmar parallel corpus based on multidimensional evaluation.

[0021] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the program to implement the automatic construction method of the large model China-Myanmar parallel corpus based on multidimensional evaluation.

[0022] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the automatic construction method of the large-scale China-Myanmar parallel corpus based on multidimensional evaluation.

[0023] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the automatic construction method of the large-scale Chinese-Myanmar parallel corpus based on multidimensional evaluation.

[0024] The beneficial effects of this invention are: it can construct high-quality Chinese-Myanmar parallel corpora, significantly improving machine translation performance on low-resource language pairs. This method uses a multi-dimensional evaluation mechanism to filter the generated pseudo-parallel corpora, thereby improving data quality and ultimately providing high-quality training data for the machine translation model. The corpus constructed by this invention can effectively improve the model's translation performance, significantly enhancing its translation quality in low-resource Chinese-Myanmar language environments. Attached Figure Description

[0025] Figure 1 This is the overall flowchart of the present invention. Detailed Implementation

[0026] Example 1: As Figure 1 As shown, the specific steps of the automatic construction method for large-scale Chinese-Myanmar parallel corpora based on multidimensional evaluation are as follows:

[0027] Step 1: Prepare candidate datasets. Collect parallel English-Burmese bilingual corpora and construct parallel candidate datasets for English-Burmese-Chinese trilingual corpora through large-scale model translation;

[0028] The specific steps of Step 1 are as follows:

[0029] Step 1.1: 291,079 Burmese-English parallel corpora were collected from the Hugging Face website to create candidate datasets;

[0030] Step 1.2: Use the LLaMA 3.1-8b model to translate the collected English-Myanmar parallel corpus and construct a parallel candidate dataset of English, Burmese and Chinese.

[0031] Step 1.3: After initial data cleaning, remove noisy data such as blank lines and incorrect language to ensure the quality of the candidate dataset.

[0032] Step 2: Calculate semantic similarity to filter candidate sets and select high-quality English-Chinese-Myanmar parallel corpus datasets;

[0033] The specific steps of Step 2 are as follows:

[0034] Step 2.1: Calculate the semantic similarity between each team of candidate data (e.g., Chinese and Burmese, Chinese and English) using cosine similarity. The specific calculation formula is as follows:

[0035]

[0036] Where A and B are vector representations of two texts. and These are the Euclidean norms of vectors A and B, respectively.

[0037] Step 2.2: Using a weighted average strategy, the similarity of each candidate corpus pair is calculated using weighted averages. High-quality English-Chinese-Myanmar parallel corpus datasets are selected based on the comprehensive semantic similarity value. Corpus pairs with similarity below 0.7 are removed, and corpus pairs with higher similarity are selected to form a high-quality candidate dataset. The specific weighted average formula is as follows:

[0038]

[0039] in , , These are the weight values, which are as follows in this method: , ,

[0040] Step 3: Construct large-scale model prompt words and score the trilingual parallel corpus using the large-scale model;

[0041] The specific steps of Step 3 are as follows:

[0042] Step 3.1: Design a simple and efficient input template for a large language model system based on task orientation. By providing a clear structure and guiding prompts, the template can effectively reduce the output deviation of the model, thereby improving the accuracy of task execution.

[0043] Step 3.2: Use the LLaMA 3.1-8b model to score the trilingual parallel corpus. The scoring criteria include translation accuracy, grammatical structure, and language fluency. Specific scoring instructions are as follows:

[0044] You are a parallel corpus quality evaluator. Here are three sentences:

[0045] Chinese: {zh} English: {en} Burmese: {my}

[0046] Please rate the following aspects (0-5 points):

[0047] 1. Chinese-English semantic consistency

[0048] 2. Semantic consistency between Chinese and Burmese

[0049] 3. Semantic consistency between English and Burmese

[0050] 4. Overall consistency across the three languages

[0051] 5. Fluency in Chinese

[0052] 6. Fluency in English

[0053] 7. Fluency in Burmese

[0054] Please strictly use JSON format for output, for example:

[0055] {{"zh-en":4,"zh-my":3,"en-my":4,"overall":3,"fluency":{{"zh":5,"en":4,"my":3}}}}.

[0056] Step 4: Rank the corpora according to the comprehensive scores, select high-quality data to construct a high-quality parallel corpus of Chinese and Burmese.

[0057] The specific steps of Step 4 are as follows:

[0058] Step 4.1: Sort the candidate corpora according to the comprehensive score, and select the corpora with higher scores as the final high-quality China-Myanmar parallel corpus.

[0059] Experiment 1: Comparison between baseline experiment and incremental training with pseudo-corpus:

[0060] This experiment first uses a baseline model as a control. The baseline model is a Transformer model trained on a parallel corpus of 550,000 Chinese and Burmese linguistic samples, trained using the fairseq framework, and ultimately achieving a BLEU score of 16.65. Next, comparative experiments are conducted using different pseudo-corpus generation methods and incremental training schemes.

[0061] Table 1 Experimental Results

[0062] Back-translation data augmentation: After incremental training with 300,000 pseudo-corpora generated through back-translation data augmentation, the BLEU score increased to 16.83, an improvement of 0.18 compared to the baseline model's 16.65. This improvement indicates that back-translation data augmentation has a certain positive effect on the model's translation performance, but the increase is relatively limited due to the noise that may be included in the back-translated data.

[0063] Incremental Training with High-Quality Pseudo-corpora: When using high-quality pseudo-corpora generated by this method for incremental training, the BLEU score is significantly improved. With 5000 high-quality pseudo-corpora, the BLEU score reaches 17.54, an improvement of 0.71 compared to back-translation augmentation; while with 50,000 high-quality pseudo-corpora, the BLEU score further increases to 17.72, demonstrating the significant performance improvement effect of generating more high-quality pseudo-corpora.

[0064] Mixing high-quality and low-quality pseudo-corpora: With a mixture of high-quality and low-quality pseudo-corpora, although the corpus size increased to 100,000 entries, the BLEU score improved to 17.25, a slight decrease compared to training with only high-quality pseudo-corpora. It can be inferred that the introduction of low-quality pseudo-corpora may have had a negative impact on model training; while increasing the corpus size, it does not necessarily improve translation quality effectively.

[0065] This experiment concludes that incremental training using high-quality pseudo-corpora generated by this method significantly improves the model's BLEU score compared to traditional back-translation data augmentation methods, especially when the amount of high-quality pseudo-corpora is large. However, when using a mixture of high-quality and low-quality pseudo-corpora, the increased corpus size does not significantly improve model performance, and may even slightly reduce performance in some cases. This indicates that the quality of the pseudo-corpora is a key factor affecting model performance during incremental training.

[0066] The specific embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. An automatic construction method for a large-scale model of Myanmar-China parallel corpus based on multidimensional evaluation, characterized by: The specific steps of the method for automatically constructing a large-scale parallel corpus of Chinese and Burmese languages ​​based on multidimensional evaluation are as follows: Step 1: Prepare candidate datasets; collect parallel English-Myanmar bilingual corpora, and construct parallel candidate datasets for English-Myanmar-Chinese trilingual corpora through large-scale model translation. Step 2: Calculate semantic similarity to filter candidate sets and select English-Chinese-Myanmar parallel corpus datasets; Step 3: Construct large-scale model prompt words and score the trilingual parallel corpus using the large-scale model; Step 4: Rank the corpora according to the comprehensive scores, select high-quality data to construct a high-quality parallel corpus of Chinese and Burmese.

2. The method for automatically constructing a large-scale parallel corpus of Chinese and Burmese languages ​​based on multidimensional evaluation according to claim 1, characterized in that: The specific steps of Step 1 are as follows: Step 1.1: Collect Burmese-English parallel corpora from the Hugging Face website to create candidate datasets; Step 1.2: Use the LLaMA 3.1-8b model to translate the collected English-Myanmar parallel corpus and construct a parallel candidate dataset of English, Burmese and Chinese. Step 1.3: After initial data cleaning, noisy data is removed to ensure the quality of the candidate dataset.

3. The method for automatically constructing a large-scale parallel corpus of Chinese and Burmese languages ​​based on multidimensional evaluation according to claim 1, characterized in that: The specific steps of Step 2 are as follows: Step 2.1: Calculate the semantic similarity between each team of candidate data using cosine similarity. Step 2.2: Using a weighted average strategy, the similarity of each pair of candidate corpora is calculated by weighting, and high-quality English-Chinese-Myanmar parallel corpus datasets are selected based on the comprehensive semantic similarity value.

4. The method for automatically constructing a large-scale parallel corpus of Chinese and Burmese languages ​​based on multidimensional evaluation according to claim 1, characterized in that: The specific steps of Step 3 are as follows: Step 3.1: Design a simple and efficient input template for a large language model system based on task orientation, by providing a clear structure and guiding prompts; Step 3.2: Use the LLaMA 3.1-8b model to score the trilingual parallel corpus. The scoring criteria include translation accuracy, grammatical structure, and language fluency.

5. An automatic construction system for a large-scale Chinese-Myanmar parallel corpus based on multidimensional evaluation, characterized in that: The system includes a module for performing the automatic construction method of large-scale Myanmar-China parallel corpus based on multidimensional evaluation as described in any one of claims 1 to 4.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that, When the processor executes the program, it implements the automatic construction method for large-scale Myanmar-China parallel corpus based on multidimensional evaluation as described in any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the automatic construction method for large-scale Myanmar-China parallel corpus based on multidimensional evaluation as described in any one of claims 1 to 4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the automatic construction method for large-scale Myanmar-China parallel corpus based on multidimensional evaluation as described in any one of claims 1 to 4.

Images