Low-resource language machine translation method and device based on multilingual semantic understanding driving

Abstract

The application discloses a kind of low-resource language machine translation method and device based on multilingual semantic understanding driving, method includes the following steps: constructing data set and pre-processing data set;Build semantic representation model, use the paraphrase sentence in paraphrase corpus to train semantic representation model, the vectorization representation of target language monolingual corpus is carried out, and semantic retrieval library is established;Mask language modeling target and inter-sentence similarity constraint are introduced, and enhanced semantic retrieval model is obtained;With enhanced semantic retrieval model, retrieve the candidate sentence similar to target language sentence semantics in semantic retrieval library, and carry out sorting screening, retain Top-K candidate sentence;After screening, candidate sentence is respectively paired with source language sentence, and enhanced parallel corpus is constructed and merged with original parallel corpus, to form extended training set;Neural machine translation model is trained using extended training set, and output translation text.The method of the application can improve semantic consistency, translation quality and cross-language generalization ability.

Description

Technical Field

[0001] This invention relates to the field of machine translation, and in particular to a low-resource language machine translation method and apparatus driven by multilingual semantic understanding. Background Technology

[0002] In recent years, data augmentation methods have been widely used to alleviate the problem of insufficient training data in low-resource machine translation. Existing methods typically generate pseudo-parallel data through back-translation, random replacement, deletion, reordering, label-based augmentation, or by using external translation tools, in order to expand the training sample size and improve machine translation performance. However, the pseudo-parallel samples generated by existing augmentation methods are of unstable quality and are prone to introducing problems such as grammatical errors, semantic shifts, and contextual inconsistencies. Especially in low-resource language scenarios, the capabilities of the basic translation model itself are limited. When using it to further generate augmented samples, the original errors are often propagated and amplified, resulting in high noise in the augmented data and even negative transfer to the training of the translation model. Summary of the Invention

[0003] This invention aims to address at least one of the technical problems existing in the prior art. To this end, this invention proposes a low-resource language machine translation method driven by multilingual semantic understanding. This method can effectively utilize semantic knowledge from target language monolingual corpora under conditions of scarce parallel corpora, thereby improving the semantic consistency, translation quality, and cross-linguistic generalization ability of low-resource machine translation models.

[0004] The present invention also proposes an apparatus having the above-mentioned low-resource language machine translation method driven by multilingual semantic understanding.

[0005] The low-resource language machine translation method based on multilingual semantic understanding driven by the first aspect of the present invention includes the following steps: A dataset is constructed and preprocessed, the dataset including paraphrasing corpus, target language monolingual corpus, and original parallel corpus; Construct a semantic representation model for semantic retrieval of the target language, train the semantic representation model using the explanatory sentences in the explanatory corpus, vectorize the target language monolingual corpus, and establish a semantic retrieval library; By introducing masked language modeling objectives and inter-sentence similarity constraints during the training process of the semantic representation model, an enhanced semantic retrieval model is obtained. Using the target language sentence in the original parallel corpus as the query, the enhanced semantic retrieval model retrieves candidate sentences that are semantically similar to the target language sentence in the semantic retrieval library, and sorts and filters them according to similarity scores, retaining the Top-K candidate sentences; The selected candidate sentences are re-paired with the corresponding source language sentences in the original parallel corpus to construct an enhanced parallel corpus. The enhanced parallel corpus is then merged with the original parallel corpus to form an extended training set. The neural machine translation model is trained using the extended training set, and the translated text is output.

[0006] According to embodiments of the present invention, a low-resource language machine translation method based on multilingual semantic understanding is proposed. This method continuously trains a multilingual pre-trained language model using paraphrased sentences. During the training of the semantic representation model, masked language modeling objectives and inter-sentence similarity constraints are introduced to enhance the semantic retrieval model, thereby improving its ability to model semantic equivalence and near-synonymous relationships. Furthermore, by constructing enhanced parallel corpora and using them for machine translation model training, the method effectively utilizes semantic knowledge from the target language monolingual corpus under conditions of scarce parallel corpora, thereby improving the semantic consistency, translation quality, and cross-lingual generalization ability of the low-resource machine translation model.

[0007] Optionally, the preprocessing method for the dataset includes: cleaning, integrating, randomly shuffling and uniformly formatting the target language monolingual corpus, and performing sub-word segmentation on the original parallel corpus.

[0008] Optionally, the overall training objective of the constructed semantic representation model is: Where λ is a hyperparameter that controls the weights of the loss in masked language modeling. For masked language modeling loss, This represents the loss of sentence similarity.

[0009] Optionally, the method for calculating the mask language modeling loss includes: in, and Let x and y represent the sets of words that are masked, respectively. and Let represent the corresponding sets of observable words, and θ represent the parameters of the pre-trained language model.

[0010] Optionally, the method for calculating the inter-sentence similarity loss includes: Where B represents the set of other paraphrased sentences besides y in the current training batch, and sim(x, y) represents the semantic similarity between sentences x and y.

[0011] Optionally, the hyperparameter λ may take different values ​​in semantic retrieval tasks of different languages.

[0012] Optionally, for English semantic retrieval tasks, λ is set to 0.1, and for Chinese semantic retrieval tasks, λ is set to 0.3.

[0013] Optionally, the number of candidate sentences K is 3.

[0014] According to a second aspect of the present invention, the machine translation apparatus includes at least one processor, at least one memory, and computer program instructions stored in the memory, which, when executed by the processor, implement the above-described low-resource language machine translation method driven by multilingual semantic understanding.

[0015] The machine translation apparatus according to the present invention can perform a low-resource language machine translation method driven by multilingual semantic understanding, thereby improving the semantic consistency, translation quality and cross-language generalization ability of the low-resource machine translation apparatus. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart of a low-resource language machine translation method based on multilingual semantic understanding according to the present invention; Figure 2 This is a comparison chart of the average CHRF results of the method of this invention and various benchmark methods on low-resource machine translation tasks; Figure 3 This is a comparison chart of the method of this invention and various benchmark methods in terms of enhancing data quality; Figure 4 This is a graph showing the performance difference of the method of the present invention in English and Chinese semantic retrieval tasks under two settings: without similarity loss and with similarity loss. Figure 5 This is a line graph showing the semantic retrieval performance of the method of the present invention under different hyperparameters λ; Figure 6 This is a schematic diagram illustrating the differences between the method of the present invention and conventional enhancement methods; Figure 7 This is a schematic diagram of the overall process of the method of the present invention. Detailed Implementation

[0018] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0019] With the increasing demands for globalization, cross-lingual information dissemination, and intelligent language services, low-resource language machine translation has become a key foundational technology in education, public services, cross-cultural communication, and multilingual information acquisition. However, compared to high-resource languages, low-resource languages ​​have long faced practical constraints such as a scarcity of bilingual parallel corpora, insufficient domain coverage, and difficulties in constructing language resources. These constraints make it difficult for neural machine translation models to fully learn stable and reliable cross-lingual mapping relationships, resulting in limited translation quality and insufficient generalization ability. In particular, under conditions of limited training data, problems such as incomplete semantic expression, insufficient context adaptation, and unstable translations are more likely to occur.

[0020] In recent years, data augmentation methods have been widely used to alleviate the problem of insufficient training data in low-resource machine translation. Existing methods typically generate pseudo-parallel data through back-translation, random replacement, deletion, reordering, label-based augmentation, or by using external translation tools, in order to expand the scale of training samples and improve machine translation performance. While these methods have improved the problem of scarce training data in low-resource scenarios to some extent, they still have several significant shortcomings in practical applications.

[0021] First, the pseudo-parallel samples generated by existing augmentation methods are of unstable quality and are prone to introducing problems such as grammatical errors, semantic shifts, and inconsistencies in context. Especially in low-resource language scenarios, the capabilities of the basic translation model itself are limited. When using it to generate augmented samples, the original errors are often propagated and amplified, resulting in high noise in the augmented data and even negative transfer to the training of the translation model.

[0022] Secondly, existing methods are heavily reliant on external translation tools or complex generation processes. Whether it is a back-translation method or a method that constructs pseudo-parallel sentence pairs based on a third-party system, it usually requires additional calls to translation models, alignment tools, or multi-stage preprocessing processes, resulting in high training costs, high implementation complexity, and limiting the transferability and deployability of the methods in more low-resource language scenarios.

[0023] Third, existing data augmentation methods do not adequately utilize the semantic knowledge inherent in large-scale monolingual corpora of the target language. Abundant target language monolingual resources contain rich expression patterns, semantic proximity relationships, and natural language variations. This information is invaluable for improving the target language generation and semantic expression capabilities of low-resource machine translation models. However, many existing methods primarily focus on directly generating pseudo-parallel data, lacking effective identification and screening mechanisms to determine "which monolingual sentences are semantically close to the original target sentence and which candidate samples are worth retaining for augmentation training." Therefore, it is difficult to fully tap the potential value of target language monolingual resources.

[0024] Fourth, while existing semantic representation methods have achieved good results in tasks such as natural language understanding, semantic retrieval, and text representation learning, their integration in low-resource machine translation data augmentation scenarios remains insufficient. On the one hand, multilingual pre-trained language models typically possess strong general representation capabilities, but their original training objectives may not be directly adaptable to the sentence-level semantic similarity modeling capabilities required for low-resource machine translation. On the other hand, existing methods often lack mechanisms to tightly couple semantic representation learning with downstream translation augmentation tasks, making it difficult for semantic retrieval models to accurately identify the most suitable target language candidate sentences for constructing augmented parallel samples.

[0025] Fifth, existing enhancement sample selection mechanisms are relatively crude, and sentences with low relevance, weak relevance, or even semantic deviation in the candidate samples are prone to entering the enhancement training set. If there is a lack of effective ranking and selection based on semantic similarity, and retention is based solely on superficial similarity or simple rules, noisy samples are easily introduced into the model training process, weakening the actual benefits of enhancement methods and further affecting the stability and generalization performance of low-resource translation systems.

[0026] Furthermore, low-resource machine translation systems exhibit significant performance variations across different language directions and target language scenarios. Existing technologies often struggle to maintain consistent and effective enhancement results across multiple languages, regions, and target language settings. Particularly in the absence of sufficient parallel resources, how to construct semantically consistent, naturally expressed, and directly usable augmented parallel samples using target language monolingual resources remains a crucial technical challenge that urgently needs to be addressed in the field of low-resource machine translation.

[0027] Based on the problems existing in the prior art, and referring to Figure 1 This invention proposes a low-resource language machine translation method driven by multilingual semantic understanding, comprising the following steps: S1: Construct and preprocess the dataset, which includes a semantic corpus, a target language monolingual corpus, and original parallel corpora. The dataset used is a publicly released standard data resource, covering both semantic retrieval and low-resource machine translation tasks. The semantic retrieval part is trained and evaluated using English and Chinese semantic datasets; the machine translation part covers 11 low-resource languages ​​and is tested under XX-En and XX-Zh translation settings to verify the stability and generalization ability of the method in multilingual, multi-regional, and different target language scenarios.

[0028] S2: Construct a semantic representation model for target language semantic retrieval. Train the semantic representation model using explanatory sentences from the explanatory corpus, and vectorize the target language monolingual corpus to establish a semantic retrieval database. Specifically, using explanatory sentences to train the semantic representation model allows sentences to be mapped to a unified semantic space; vectorizing the target language monolingual corpus to establish a queryable semantic retrieval database provides a foundation for subsequent retrieval of semantically similar candidate sentences.

[0029] S3: By introducing masked language modeling objectives and inter-sentence similarity constraints during the training process of the semantic representation model, an enhanced semantic retrieval model is obtained. This enhances the semantic representation model's ability to model semantic equivalence relations, synonymous expressions, and sentence-level semantic similarity.

[0030] S4: Using the target language sentence in the original parallel corpus as the query, an enhanced semantic retrieval model is used to retrieve candidate sentences that are semantically similar to the target language sentence from the semantic retrieval database. These candidate sentences are then sorted and filtered based on similarity scores, retaining the Top-K candidate sentences. This approach retains the candidate sentences that are semantically closest to the target language sentence, reducing noise interference from low-relevance samples and semantically offset samples.

[0031] S5: The selected candidate sentences are re-paired with their corresponding source language sentences in the original parallel corpus to construct an enhanced parallel corpus. This enhanced parallel corpus is then merged with the original parallel corpus to form an expanded training set. Thus, it is possible to effectively expand the training samples for low-resource machine translation without relying on additional manually labeled parallel data.

[0032] S6: Train the neural machine translation model using an expanded training set and output the translation.

[0033] According to embodiments of the present invention, a low-resource language machine translation method driven by multilingual semantic understanding addresses the problems of insufficient training data, semantic distortion of augmented samples, and unstable translation performance in low-resource machine translation through a process of "continuous training of paraphrased corpus—target-side semantic retrieval—candidate ranking and screening—construction of augmented parallel corpus—machine translation training." This method not only retrieves semantically similar natural expressions from the target language monolingual corpus but also controls the quality of augmented samples through semantic ranking and screening mechanisms, thereby suppressing common problems in traditional pseudo-parallel data construction methods such as grammatical distortion, semantic drift, and noise accumulation. Compared with data augmentation methods that rely on external translation systems or random editing, the present invention uses an augmented semantic retrieval model as its core support, directly retrieving highly relevant candidate sentences in the target language semantic space. This more stably preserves natural, accurate, and semantically consistent expressions in the target language, reducing interference from irrelevant and low-quality samples on machine translation training.

[0034] This invention presents a low-resource language machine translation method driven by multilingual semantic understanding. It uses a large-scale monolingual corpus of the target language as the source of semantic knowledge. First, it continuously trains a multilingual pre-trained language model using paraphrased sentences. Beyond masking the language modeling objective, it further introduces inter-sentence similarity constraints to enhance the model's ability to model semantic equivalence and near-synonymous relationships. Using target language sentences from the original parallel corpus as queries, it retrieves semantically similar candidate sentences from the target language monolingual corpus and retains the most relevant samples by similarity ranking. Finally, it re-pairs the retained target language candidate sentences with the original source language sentences to construct... This approach enhances parallel corpora for machine translation model training, thereby improving the learning ability of low-resource translation models to natural expressions and semantic consistency of the target language without relying on additional manually labeled parallel data and complex external translation tools. It reduces the dependence of traditional data augmentation methods on external translation tools and complex generation processes, reduces semantic noise and inconsistency issues in augmented samples, and improves the accuracy, stability, and generalization ability of machine translation systems in multilingual, multi-target language, and multi-regional scenarios. It overcomes the shortcomings of existing low-resource machine translation augmentation methods, such as unstable sample quality, insufficient semantic utilization, and limited augmentation benefits.

[0035] In step S1, the preprocessing methods for the dataset include: cleaning, integrating, randomly shuffling and uniformly formatting the target language monolingual corpus, and performing word segmentation on the original parallel corpus.

[0036] In step S2, a semantic representation model for semantic retrieval of the target language is constructed. Specifically, a set K of sentence pairs is constructed using a paraphrased corpus, where each sentence pair...<x, y> This represents two semantically similar or equivalent sentences. During the continuous training of the multilingual pre-trained language model, masked language modeling objectives and inter-sentence similarity constraints are introduced to enhance the model's ability to model sentence-level semantic similarity relationships. The overall training objective of the constructed semantic representation model is: Where λ is a hyperparameter that controls the weights of the loss in masked language modeling. For masked language modeling loss, This represents the loss of sentence similarity.

[0037] In step S3, the method for calculating the masked language modeling loss includes: in, and Let x and y represent the sets of words that are masked, respectively. and Let represent the corresponding sets of observable words, and θ represent the parameters of the pre-trained language model.

[0038] In step S3, the method for calculating the inter-sentence similarity loss includes: Where B represents the set of other paraphrased sentences besides y in the current training batch, and sim(x, y) represents the semantic similarity between sentences x and y.

[0039] The calculation method is as follows: Where h(·) represents the vector representation of the sentence. Through the above joint training, this invention obtains an enhanced semantic retrieval model, which is used to retrieve natural expressions that are semantically close to the original target sentence in the target language monolingual corpus.

[0040] In step S4, the target language sentence in the original parallel corpus is used as the query. The enhanced semantic retrieval model is used to retrieve candidate sentences that are semantically similar to the target language sentence in the semantic retrieval library. The candidate sentences are sorted and filtered according to the similarity score, and the Top-K candidate sentences are retained.

[0041] For example, the original parallel corpus sentence pairs can be represented as<Src, Trg> Using the target language sentence Trg as the query, an enhanced semantic retrieval model is used to retrieve the top-k sentences with the highest semantic similarity from the semantic retrieval database, for example, Trg'_1, ..., Trg'_k. The value of K is not limited; for example, K is set to 3. Subsequently, these retrieved target language sentences can be re-paired with the original source language sentence Src to construct enhanced parallel sentence pairs, for example, ...<Src, Trg'_1> ,<Src, Trg'_2> , ...,<Src, Trg'_k> Then, the constructed enhanced parallel sentence pairs are merged with the original parallel corpus C to form the enhanced parallel corpus C'. Based on this, the training objective of the machine translation model can be expressed as: in, Let |C'| represent the parameters of the machine translation model, and |C'| represent the size of the augmented parallel corpus. Therefore, this invention does not rely on additional manually labeled parallel data, nor does it require complex external translation systems to generate pseudo-samples. Instead, it directly utilizes semantically nearest sentences from the target language monolingual resources to construct augmented training data, thereby improving the training quality and generalization ability of low-resource machine translation models.

[0042] This invention conducts comparative experiments on multiple internationally published standard evaluation settings to systematically verify its effectiveness, robustness, and generalization ability in semantic retrieval and low-resource machine translation tasks. The evaluation data consists of two parts: First, the semantic retrieval evaluation uses publicly available English and Chinese semantic datasets. For English, ParaSCI and Quora datasets are used, while for Chinese, PKU Paraphrase Ban uses subsets of BQ, LCQMC, and PAWS from the LUG dataset. Top-3, Top-5, Top-10, Top-20, and Top-30 accuracy rates are used to evaluate semantic retrieval performance. Second, the low-resource machine translation evaluation covers 11 low-resource languages ​​and is tested under XX-En and XX-Zh translation settings. The Middle Eastern region includes Azerbaijani, Persian, Hungarian, and Hebrew; the South Asian region includes Urdu, Bengali, and Hindi; and the Southeast Asian region includes Indonesian, Malay, Vietnamese, and Thai. The performance evaluation metrics adopt the automatic evaluation metric system commonly used in the field of machine translation, including SacreBLEU, CHRF, METEOR and TER, and further combine BERTScore, ROUGE and perplexity metrics to comprehensively measure the quality of enhanced samples and translations from multiple levels such as semantic consistency, sufficiency and fluency.

[0043] Regarding evaluation metrics, this invention uses Top-{3,5,10,20,30} accuracy to evaluate the semantic retrieval model; for the machine translation model, SacreBLEU is mainly used to evaluate translation quality, and CHRF, METEOR, and TER are further combined to evaluate the robustness of the methods; at the same time, to verify the quality of the augmented samples, this invention also uses BERTScore, ROUGE-{1,2,L}, and perplexity PPL to comprehensively evaluate the semantic consistency, sufficiency, and fluency of the augmented data.

[0044] The specific information regarding the models and methods used in this invention for comparison with Enhanced Semantic Retrieval Models (OURS) is as follows: BERT: BERT is a language representation model that is pre-trained based on masked language modeling and next sentence prediction tasks. This invention uses it as the basic representation learning baseline in semantic retrieval tasks to measure the performance of general pre-trained models in sentence semantic matching.

[0045] SimCSE: SimCSE is a model that trains sentence vector representations through contrastive learning. This invention uses it as a semantic retrieval baseline to compare the performance of self-supervised sentence representation methods in semantically similar sentence retrieval tasks.

[0046] InfoXLM: InfoXLM is a multilingual pre-trained model that combines masked language modeling, translational language modeling, and contrastive learning. This invention uses it as a strong baseline model for multilingual semantic retrieval.

[0047] VECO: VECO is a multilingual pre-trained language model that can be adapted to both natural language understanding and generation tasks. This invention uses it as a baseline for cross-lingual semantic representation and retrieval.

[0048] CMLM: CMLM is an unsupervised sentence representation model based on conditional mask language modeling training. This invention uses it as a semantic retrieval baseline to compare the performance differences of different sentence vector modeling methods.

[0049] LaBSE: LaBSE is a language-independent representation model for multilingual sentence embedding learning. This invention uses it as a representative strong baseline in cross-lingual semantic retrieval tasks.

[0050] BASE: BASE stands for machine translation model trained solely on raw parallel corpora without any data augmentation methods, serving as a baseline contrast system for low-resource machine translation tasks.

[0051] FT: FT is a data augmentation method that augments training samples by translating source language monolingual data to the target language side. It is used to compare the effectiveness of source-side monolingual augmentation strategies.

[0052] BT: BT is a classic back-translation data augmentation method that constructs pseudo-parallel corpora by translating monolingual data of the target language back into the source language, in order to improve the performance of low-resource machine translation.

[0053] TAGGEDBT: TAGGEDBT adds explicit labels before back-translation of data to distinguish between original samples and augmented samples, thereby reducing the interference of pseudo-parallel data on model training.

[0054] IBT: IBT is an iterative back-translation method that gradually improves the performance of low-resource translation models by constructing and utilizing pseudo-parallel corpora in multiple rounds.

[0055] DROP: DROP is a method for constructing augmented samples by randomly deleting words from sentences. It is used to evaluate the role of simple perturbation-based data augmentation in low-resource machine translation.

[0056] BLANK: BLANK is an enhancement method that replaces parts of a word with blank placeholders to improve the robustness of a model by perturbing the input.

[0057] SWAP: SWAP is a data augmentation method that randomly swaps word order within a local window to construct perturbative training samples and enhance the model's adaptability to word order changes.

[0058] TARGET AUG: TARGET AUG is a data augmentation method that only enhances the target language side, used to compare the impact of target-side augmentation strategies on machine translation performance.

[0059] REVERSIBLE AUG: REVERSIBLE AUG is a reversible data augmentation method that balances symbolic diversity and semantic consistency. It is used as a representative comparative baseline for neural machine translation data augmentation strategies in recent years.

[0060] Table 1: Comparison of English semantic search results

[0061] Table 1 shows the comparison results of the method of this invention with several baseline methods on the English semantic retrieval task. The evaluation metrics are Top-3, Top-5, Top-10, Top-20, Top-30, and average accuracy. The results show that the method of this invention achieves the highest results in all metrics, indicating that it has stronger semantic representation and retrieval capabilities in the English semantically similar sentence retrieval task.

[0062] Table 2 Comparison of Chinese Semantic Retrieval

[0063] Table 2 shows the comparison results between the method of this invention and various baseline methods on the Chinese semantic retrieval task. The evaluation metrics are Top-3, Top-5, Top-10, Top-20, Top-30, and average accuracy. The results show that the method of this invention significantly outperforms various baseline models on the Chinese semantic retrieval task, indicating that the method has good stability and generalization ability in different target language scenarios.

[0064] Table 3. Comparison of SacreBLEU scores for the XX-En translation task

[0065] Table 3 shows the SacreBLEU scores of the proposed method and various benchmark models on the XX-En translation task, covering multiple low-resource language directions in the Middle East, South Asia, and Southeast Asia. Experimental results demonstrate that the proposed method achieves optimal or better translation performance in all language directions, indicating that the proposed semantic retrieval-driven data augmentation strategy can consistently improve translation quality in low-resource language to English translation tasks and possesses good cross-linguistic generalization ability.

[0066] Table 4. Comparison of SacreBLEU scores for the XX-Zh translation task

[0067] Table 4 shows the comparison results of the SacreBLEU scores of the method of this invention and various benchmark models on the XX-Zh translation task, covering multiple low-resource language directions in the Middle East, South Asia, and Southeast Asia. Experimental results show that the method of this invention also achieves optimal performance in each language direction, indicating that this invention is not only applicable to low-resource translation scenarios with English as the target language, but also maintains good enhancement effects and stable performance improvements when Chinese is the target language, demonstrating strong cross-target language adaptability.

[0068] Reference Figure 2 , Figure 2 The paper presents the average CHRF comparison results between the method of this invention and various benchmark methods on low-resource machine translation tasks, whereby... Figure 2 (a) Corresponding to the XX-En translation task. Figure 2(b) Corresponding to the XX-Zh translation task. Experimental results show that the method of this invention achieves the highest average CHRF score in both translation settings, outperforming BASE, FT, BT, TAGGEDBT, IBT, and various random perturbation and target-side augmentation methods. This demonstrates that the present invention has significant advantages in improving the character-level matching quality of the translated text, maintaining the stability of the translation results, and enhancing the robustness of low-resource translation. These results further verify that the semantic retrieval-based data augmentation strategy proposed in this invention can stably improve translation quality under different target language conditions.

[0069] Reference Figure 3 , Figure 3 The comparison results of the method of this invention with various benchmark methods in terms of improving data quality are presented, wherein Figure 3 (a) shows the BERTScore comparison results. Figure 3 (b) shows the comparison results for ROUGE-L. Figure 3 (c) Comparison results of perplexity level (PPL). Experimental results show that the method of this invention achieves the highest results on BERTScore and ROUGE-L metrics, while achieving better or lower results on the PPL metric. This indicates that the enhanced samples generated by this invention outperform various benchmark methods in terms of semantic consistency, content sufficiency, and language fluency. These results demonstrate that this invention not only improves the final machine translation performance but also reduces semantic shift, expression distortion, and noise accumulation from the source of enhanced data, thereby providing higher-quality training samples for low-resource machine translation models.

[0070] In addition, to verify the effectiveness of key components, ablation and sensitivity experiments were conducted on the similarity loss and hyperparameter λ in the multilingual semantic representation model.

[0071] like Figure 4 As shown, the performance differences between "without similarity loss" and "with similarity loss" settings were compared on English and Chinese semantic retrieval tasks. Evaluation metrics included Top-3, Top-5, Top-10, Top-20, and Top-30 accuracy. The results show that, on both English and Chinese tasks, the introduction of similarity loss significantly outperformed the setting without it in all retrieval metrics. This indicates that similarity loss effectively enhances the model's ability to model semantic equivalence and near-synonymous expressions, making semantically similar sentences closer in the representation space, thereby significantly improving semantic retrieval performance.

[0072] In summary, the method proposed in this invention significantly outperforms various strong baseline models in both English and Chinese semantic retrieval tasks, and achieves more stable performance improvements in both XX-En and XX-Zh low-resource machine translation tasks. Compared to traditional back-translation, tag-based back-translation, random perturbation-based augmentation, and methods that only augment the target side, this invention can more effectively retrieve natural expressions semantically close to the original target sentence from the target language monolingual corpus, reduce semantic shifts, grammatical distortions, and noise interference in augmented samples, and improve the usability of augmented parallel corpora and the training effect of machine translation models. Therefore, this invention has better translation quality improvement capabilities, stronger cross-language generalization capabilities, and higher engineering practical value in low-resource scenarios.

[0073] like Figure 5 As shown, the impact of the hyperparameter λ on semantic retrieval performance was examined, where λ controls the weight of the masked language modeling loss in the overall objective function. Experimental results show that the optimal value of λ varies for different language tasks: for English semantic retrieval, the model achieves the best average performance when λ = 0.1; for Chinese semantic retrieval, the model achieves the best average performance when λ = 0.3. Meanwhile, the overall performance fluctuation of the model is relatively limited under different λ values, indicating that the method of this invention has good training stability; however, a reasonable selection of λ can still help to further balance language modeling ability and semantic similarity modeling ability, thereby obtaining better retrieval results. Therefore, in subsequent experiments, this invention adopts a more optimal λ setting for different target languages ​​to improve the overall system performance.

[0074] For example, Figure 6 This paper illustrates the differences between conventional augmentation methods and the semantic retrieval-based data augmentation method proposed in this invention for low-resource machine translation data augmentation tasks. Using the input sentence "John is doing his homework carefully." as an example, it demonstrates that traditional augmentation methods are prone to introducing semantic or syntactic errors when constructing pseudo-samples, while the method of this invention can generate augmented samples that are semantically consistent with the original sentence and express naturally.

[0075] like Figure 6As shown, conventional augmentation methods typically generate augmented samples through random replacement, local editing, or word order perturbation, which easily leads to two types of problems: one is semantic errors, where the augmented sentence is similar to the original sentence in surface form, but the key content is deviated, such as incorrectly replacing "homework" with "dinner," or replacing action relationships with expressions that do not conform to the original meaning; the other is syntactic errors, where the generated result retains some original words, but the word order is disordered and the structure is incomplete, making it difficult to use as a high-quality training sample. These problems result in augmented data containing a lot of noise, which not only fails to effectively improve the performance of machine translation models but may also interfere with the training process.

[0076] In contrast, the method of this invention is based on semantic retrieval. Instead of directly subjecting the original sentence to random perturbation, it retrieves natural expressions semantically close to the original sentence from a large-scale monolingual corpus of the target language and uses these as augmentation samples. For example... Figure 6 The generated sample "John is writing his homework seriously." maintains overall semantic consistency with the original sentence, while possessing a complete syntactic structure and natural expression, making it better suited for subsequent machine translation model training. This embodiment demonstrates that the advantages of the method of the present invention are: on the one hand, it avoids the semantic drift and syntactic distortion problems common in traditional augmentation methods; on the other hand, it can utilize the natural expressions that truly exist in the target language monolingual corpus to generate higher-quality and more usable augmented samples. Therefore, the present invention can effectively reduce the noise ratio in augmented data and improve the learning ability of low-resource machine translation models to target language semantic expressions and naturally generated patterns.

[0077] For example, Figure 7 This paper presents the overall process framework of a low-resource machine translation data augmentation method and apparatus based on semantic retrieval driven by the present invention.

[0078] First, during the semantic retrieval model training phase, the system utilizes paraphrased data as its training foundation. Each data set contains semantically similar query and label sentence pairs. Through these paraphrased sentence pairs, the system trains the semantic retrieval model to learn sentence-level semantic similarity relationships and maps semantically similar but differently expressed sentences to similar representation spaces. The goal of this process is to improve the ability of multilingual pre-trained language models to model semantic equivalence and near-synonymous expressions, thereby providing a more accurate semantic representation foundation for subsequent retrieval stages.

[0079] Subsequently, in the target language monolingual retrieval stage, the system uses the target sentence Trg from the original parallel corpus as the query and retrieves multiple semantically similar candidate sentences from the target language monolingual corpus, obtaining retrieval results Trg'_1, Trg'_2, Trg'_3, ..., Trg'_n. Since these candidate sentences come from real monolingual corpora, they typically have more natural language expressions and more complete syntactic structures. Afterward, the system sorts the candidate sentences according to semantic similarity and retains the most relevant sentences based on a preset threshold or a Top-k strategy, forming the final retained samples Trg'_1, Trg'_2, ..., Trg'_k, where k is less than n. This step aims to reduce the number of low-relevance and semantically biased samples entering the augmentation dataset, thereby ensuring the overall quality of the augmented samples.

[0080] Finally, in the stage of constructing enhanced parallel corpora and training machine translation, the system re-pairs the selected target language candidate sentences with the original source language sentences to construct new enhanced parallel sentence pairs, which, together with the original parallel corpora, form an expanded training set. After training the machine translation model based on this expanded training set, the model can simultaneously utilize the alignment information in the original parallel samples and the natural expression information in the target language monolingual corpus, thereby improving translation performance, semantic consistency, and generalization ability in low-resource scenarios.

[0081] Through the above process, this invention achieves a technical solution for automatically acquiring semantically similar augmented samples from target language monolingual corpora and effectively using them for low-resource machine translation training. Compared with traditional data augmentation methods that rely on back-translation, random replacement, deletion, or reordering, this invention can more stably construct semantically consistent, naturally expressed, and syntactically complete augmented samples, reduce the interference of augmentation noise on model training, and exhibit better stability and applicability in multilingual, low-resource machine translation scenarios.

Claims

1. A low-resource language machine translation method driven by multilingual semantic understanding, characterized in that, Includes the following steps: A dataset is constructed and preprocessed, the dataset including paraphrasing corpus, target language monolingual corpus, and original parallel corpus; Construct a semantic representation model for semantic retrieval of the target language, train the semantic representation model using the explanatory sentences in the explanatory corpus, vectorize the target language monolingual corpus, and establish a semantic retrieval library; In the training process of the semantic representation model, masked language modeling objectives and inter-sentence similarity constraints are introduced to obtain an enhanced semantic retrieval model; Using the target language sentence in the original parallel corpus as the query, the enhanced semantic retrieval model retrieves candidate sentences that are semantically similar to the target language sentence in the semantic retrieval library, and sorts and filters them according to similarity scores, retaining the Top-K candidate sentences; The selected candidate sentences are re-paired with the corresponding source language sentences in the original parallel corpus to construct an enhanced parallel corpus. The enhanced parallel corpus is then merged with the original parallel corpus to form an extended training set. The neural machine translation model is trained using the extended training set and the translated text is output.

2. The low-resource language machine translation method based on multilingual semantic understanding as described in claim 1, characterized in that, The preprocessing method for the dataset includes: cleaning, integrating, randomly shuffling and uniformly formatting the target language monolingual corpus, and performing word segmentation on the original parallel corpus.

3. The low-resource language machine translation method based on multilingual semantic understanding as described in claim 1, characterized in that, The overall training objective of the constructed semantic representation model is: Where λ is a hyperparameter that controls the weights of the loss in masked language modeling. For masked language modeling loss, This represents the loss of sentence similarity.

4. The low-resource language machine translation method based on multilingual semantic understanding as described in claim 3, characterized in that, The method for calculating the masked language modeling loss includes: in, and Let x and y represent the sets of words that are masked, respectively. and Let represent the corresponding sets of observable words, and θ represent the parameters of the pre-trained language model.

5. The low-resource language machine translation method based on multilingual semantic understanding as described in claim 3, characterized in that, The methods for calculating the inter-sentence similarity loss include: Where B represents the set of other paraphrased sentences besides y in the current training batch, and sim(x, y) represents the semantic similarity between sentences x and y.

6. The low-resource language machine translation method based on multilingual semantic understanding as described in claim 3, characterized in that, The hyperparameter λ takes different values ​​in semantic retrieval tasks of different languages.

7. The low-resource language machine translation method based on multilingual semantic understanding as described in claim 6, characterized in that, For English semantic retrieval tasks, the value of λ is 0.1, and for Chinese semantic retrieval tasks, the value of λ is 0.

3.

8. The low-resource language machine translation method based on multilingual semantic understanding as described in claim 1, characterized in that, The number of candidate sentences, K, is 3.

9. A machine translation device, characterized in that, include: At least one processor, at least one memory, and computer program instructions stored in the memory, which, when executed by the processor, implement the low-resource language machine translation method based on multilingual semantic understanding as described in any one of claims 1-8.

Images