Low-resource machine translation method and system based on soft and hard retrieval and inference correction

Abstract

The application discloses a kind of based on soft and hard retrieval and inference correction extremely low resource machine translation method and system, retrieve real sentence pair from real parallel corpus by hard retrieval, and soft retrieval generates translation hypothesis to construct soft retrieval sentence pair;Two kinds of retrieval sets are formatted as enhanced context prompt text, and input into inference type large language model to compare logic, consistency check and local error correction to multiple source examples, and output answer text;Final translation is extracted by parsing again.This application combines the complementary information source of hard retrieval and soft retrieval and the inference correction ability of large language model, realizes the closed-loop optimization of " generation- reflection- correction", significantly improves the semantic accuracy and structural rationality of extremely low resource language translation, effectively solves the translation cold start and error correction problem under the condition of data scarcity.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing technology, and further relates to a method and system for extremely low-resource machine translation based on soft and hard retrieval and inference correction in the field of machine translation or language conversion technology. This invention can be used in automatic translation systems for endangered or extremely low-resource languages ​​to automatically generate preliminary translation results, thereby accelerating the processes of manual proofreading, dictionary construction, or corpus annotation. Background Technology

[0002] There are over 7,000 languages ​​in existence globally, but natural language processing (NLP) primarily focuses on fewer than 100 high-resource languages. For extremely low-resource languages ​​such as Aragon, Arana, and Asturian, the size of their parallel corpora is often less than a thousand sentences, or even completely absent. Although Meta AI's NLLB-200 model supports 200 languages, it still cannot cover all "long-tail languages," and its performance drops sharply on untrained languages. In recent years, large language models, such as GPT-4, Llama, and Qwen, have demonstrated powerful cross-language transfer capabilities, but their performance in extremely low-resource translation tasks is still limited by the quality and quantity of contextual examples.

[0003] In extremely low-resource languages ​​(ELRL) scenarios, due to the extreme scarcity of parallel corpora, traditional neural machine translation (NMT) models cannot effectively model the mapping relationship between the source and target languages, leading to a severe decline in translation quality. Existing retrieval-augmented generation (RAG) methods rely on high-quality, highly relevant external examples, which fail in ELRL due to the lack of available real parallel sentence pairs. Directly applying large language models (LMMs) for zero-shot or few-shot translation is prone to semantic bias, lexical mismatches, or grammatical errors due to the lack of prior knowledge and structured reasoning ability for rare language combinations. Existing methods struggle to effectively integrate the local language structure knowledge of neural machine translation models with external example information, making it impossible to achieve dynamic error detection and correction under conditions of high uncertainty.

[0004] Shenyang Yayi Network Technology Co., Ltd. disclosed a low-resource language translation method in its patent application "A Translation Method Applied to Low-Resource Machine Translation" (Application Date: 2022-02-24, Application No.: CN 202210172910.2, Authorization Announcement No.: CN 114580441B). This translation method introduces a multi-scale sampling strategy based on the standard multi-head self-attention mechanism. By sampling the source language input at different granularities, multiple attention weight matrices are generated and fused, thereby enhancing the model's modeling ability under low-resource conditions. This method does not increase the number of model parameters, has good versatility, can be embedded into existing Transformer architectures, and is suitable for language pairs with thousands to tens of thousands of parallel sentences. While achieving local optimization within the traditional Neural Machine Translation (NMT) framework, this method still suffers from several key limitations: First, its performance is highly dependent on a certain scale of parallel corpora, typically requiring hundreds of thousands of sentence pairs even when defined as "low-resource," making it difficult to train an effective model in extremely low-resource (<1000 sentences) or even zero-resource scenarios. Second, this method relies entirely on improvements to the model's internal structure, without incorporating external knowledge (such as bilingual dictionaries, rule systems, or multilingual priors), resulting in limited generalization ability when data is extremely scarce. Third, its end-to-end generation method lacks an error correction mechanism; once attentional biases occur due to data sparsity, inference correction is impossible. Finally, the multi-scale sampling strategy is statically embedded in the network structure, making it impossible to dynamically adjust the sampling and fusion strategies based on the semantic complexity or translation difficulty of the input sentences. These problems arise because the technical paradigm employed by this method is a typical "pure neural network architecture improvement" approach, focusing on fine-tuning the Transformer's internal attention mechanism rather than building a system-level solution. This method was applied for in 2022, a time when Large Language Models (LLM) and Retrieval Augmentation (RAG) techniques were not yet widely used in machine translation. Therefore, it did not consider leveraging the zero-shot capability of LLM or external knowledge retrieval mechanisms. Furthermore, its design goal was explicitly limited to "improving performance without increasing parameters," thus ruling out the possibility of introducing additional modules such as retrieval engines or inference engines. More importantly, this method implicitly assumes the existence of a trainable NMT model. However, for many endangered or niche languages ​​globally (such as Aragonese and Aranese), this assumption often fails, as the corpus for these languages ​​is too small to support model convergence, rendering the method ineffective in truly resource-scarce scenarios.

[0005] Hubei University disclosed a prefix-based translation method in its patent application, "A Language Translation Method Based on Prefix Tuning Machine Translation Model" (application date: 2023-02-22, patent application number: CN 202310149564.0, authorization announcement number: CN 116151278B). This method proposes a parameter efficient fine-tuning (PEFT) strategy, introducing a prefix tuning mechanism on top of a pre-trained multilingual model mBART. It uses three sets of learnable control signal matrices to respectively regulate the self-attention and cross-attention layers in the encoder and decoder. This method utilizes an MLP to map attribute labels such as word order, length, or style into lightweight prefix vectors, achieving fine-grained control of the translation output without modifying the backbone model parameters. Its core advantage lies in improving the controllability and adaptability of existing neural machine translation models in medium-to-high resource scenarios. However, this method faces fundamental limitations in scenarios with extremely low or even zero resources: it relies on supervised training data on the target language pair to optimize prefix parameters. If parallel corpora are scarce (e.g., less than 1000 sentences) or completely absent, it cannot effectively establish a mapping relationship between control signals and translation behavior, leading to the failure of the prefix module. Simultaneously, this method lacks an external knowledge fusion mechanism, cannot leverage cross-linguistic priors or examples to enhance generalization ability, and employs an end-to-end black-box generation paradigm, lacking the ability to detect and correct lexical mismatches or grammatical errors. The root cause lies in the fact that the method's initial design focused on fine-grained control under supervised training data on the target language pair—that is, optimizing translation style based on existing high-quality corpora—rather than addressing the fundamental usability issues in data-scarce scenarios. Its closed architecture and static generation process also limit its applicability in data-scarce environments.

[0006] Beijing Lanzhou Technology Co., Ltd. disclosed a vertical domain translation method in its patent application "A Vertical Domain Translation Model Training Method and Storage Medium" (application date 2022-11-17, patent application number CN 202211442764.7, authorization announcement number CN115906879 B). This method proposes a three-stage training strategy based on an encoder-decoder architecture. The three stages are sentence vector pre-training, high-resource fine-tuning, and low-resource retrieval enhancement, aiming to improve the quality of neural machine translation in vertical domains such as law and medicine. Its core lies in using high-resource vertical corpora to fine-tune the decoder (while simultaneously freezing the encoder), and combining this with semantically similar sentences retrieved from the high-resource memory to achieve cross-language multi-instance fusion and contextual proximity adjustment. This method performs well given a certain amount of high-resource parallel corpora. However, a remaining drawback is that its effectiveness highly depends on a crucial prerequisite: the existence of a sufficiently large amount of high-quality bilingual data in the vertical domain. When faced with language pairs with extremely low or even zero resources (such as minority languages ​​or endangered languages), the entire process becomes difficult to initiate, making it impossible to fine-tune the decoder or build an effective retrieval memory. A deeper problem lies in the inherently static and closed nature of the technical paradigm employed. Its retrieval mechanism relies solely on hard matching of real, high-resource sentences, lacking exploration of the model's own generative capabilities. When real examples are sparse, irrelevant, or completely absent, the enhancement effect is not only limited but may also introduce noise. Furthermore, the entire process is a deterministic feedforward structure, lacking logical reasoning, consistency checks, or error correction capabilities, making it difficult to handle semantically ambiguous or structurally complex inputs. More importantly, while freezing the encoder can prevent catastrophic forgetting, it also limits the model's potential for adaptively learning new language representations under low-resource conditions. These limitations collectively reflect that this method optimizes existing translation methods under the assumption of a certain scale of high-quality bilingual data, failing to address the translation problem in extremely low-resource scenarios where a certain scale of high-quality bilingual data is lacking.

[0007] Beijing Zhongke Wenge Technology Co., Ltd., the Institute of Automation, Chinese Academy of Sciences, and the Aviation Industry Information Center disclosed a translation method based on a large language model in their patent application, "A Training Method for a Low-Resource Language Translation Model Based on a Large Language Model" (application date: 2025-10-23, application (patent) number: CN202511521510.8, publication number: CN121351849A). This patent proposes a three-stage progressive training strategy. First, the basic large language model is pre-trained on multilingual unsupervised corpora, and multiple intermediate checkpoints are saved. Then, the model with the best verification performance is selected and fine-tuned under supervision on parallel instruction data of the low-resource language and the target language. Finally, using manually or automatically constructed preferred / rejected translation pairs, preference optimization algorithms (such as DPO or RLHF) are used to further align the model output with human preferences. This method effectively improves translation quality in low-resource language scenarios with a certain amount of parallel corpora, but its core limitation lies in its high dependence on real parallel sentence pairs; both instruction fine-tuning and preference optimization stages require such data support. This premise is often difficult to meet when dealing with languages ​​with extremely low resource availability (such as Aragonese and Aranese), as these languages ​​may have only a few hundred sentences or even lack parallel corpora entirely. Furthermore, this method employs a static training paradigm, fixing the parameters of the model once training is complete. External knowledge or examples cannot be dynamically introduced during inference, thus lacking the ability to adapt to unseen sentence structures or vocabulary. Simultaneously, if the target language is almost entirely absent from the pre-training corpus, continuing the pre-training phase will struggle to effectively model its linguistic characteristics, leading to the cold start problem. More importantly, the model lacks inference or error correction mechanisms after generating incorrect translations, making self-correction impossible. The root cause of these shortcomings lies in the fact that the patent's technical approach is based on the assumption that readily available low-resource parallel corpora exist for fine-tuning and preference learning. However, the reality is that the vast majority of the world's 7,000+ languages ​​are long-tail languages, lacking not only parallel corpora but also extremely scarce monolingual corpora. Some languages ​​are not even covered by mainstream multilingual models (such as NLLB-200). This method addresses translation capabilities by internalizing model parameters, neglecting the possibility of dynamically integrating limited external information during the inference phase. Furthermore, the entire process is confined to the training phase, with the inference process generated as a black box, making it impossible to correct inferences based on model priors and external evidence. Therefore, this approach is difficult to implement in practice when dealing with extremely low-resource languages ​​with fewer than 1000 parallel corpora, or even none. Summary of the Invention

[0008] The purpose of this invention is to address the shortcomings of the existing technologies mentioned above by proposing a machine translation method and system for extremely low-resource languages ​​based on soft and hard retrieval and inference correction. This aims to solve the three core problems of existing technologies that cannot be started, cannot correct errors, and cannot generalize under extreme data scarcity conditions, and to provide a practical machine translation path for thousands of extremely low-resource languages ​​worldwide.

[0009] The technical approach to achieving the objective of this invention is to significantly improve the translation quality and robustness of language pairs with extremely low resource availability by fusing hard retrieval examples (real examples) and soft retrieval examples (pseudo-examples generated by the model) and using a large language model with reasoning capabilities for inference correction, even when real parallel corpora are extremely scarce or even missing. This invention constructs a translation method that requires very little data, is error-correcting, and is highly universal, effectively solving the problem of existing technologies being unable to perform effective translation in extremely low resource scenarios. Addressing the common reliance on a certain scale of parallel corpora and the difficulty in handling the severe lack of bilingual data in real-world languages, this invention requires only a weak machine translation model for the target translation task and a very small (or even zero) number of real examples, significantly reducing dependence on data resources. Furthermore, existing solutions mostly employ closed, static, pure neural network architectures, which cannot introduce external knowledge or make dynamic adjustments during the generation process. This invention, however, designs an open, dynamic system architecture that organically combines high-confidence real-world examples (hard retrieval) with diverse but low-confidence pseudo-translation hypotheses (soft retrieval), forming complementary information sources. This is further enhanced by using large language models with reasoning capabilities (such as DeepSeek-R1, GPT-4o, Claude-3 Opus, etc.) to… <think>"By implementing collaborative inference, consistency verification, and local error correction under tag guidance, a closed loop of 'generation-reflection-correction' is achieved, fundamentally overcoming the inherent weakness of traditional end-to-end models where errors are irreversible. Addressing the issue of hard retrieval failing with sparse corpora, this invention innovatively proposes a soft retrieval mechanism. It treats pseudo-translations generated by weak machine translation models as valuable contextual clues, constructing structured prompts even without real-world examples, thus maintaining system usability. Furthermore, unlike existing technologies that rely on closely related languages ​​(such as using Thai to assist Lao) or intermediate languages, this invention is completely language-agnostic, without presupposing language family relationships or cross-language alignment priors. It can be directly applied to isolated languages ​​or languages ​​not covered by mainstream models, such as Basque and Aragon. More importantly, this invention abandons the cold-start steps such as fine-tuning and preference learning necessary in traditional training-driven paradigms, instead adopting an inference-driven approach. All capabilities are activated during the inference phase, truly achieving zero-sample expansion."

[0010] To achieve the above objectives, the translation method of the present invention includes the following steps:

[0011] Step 1: Use a semantic similarity function to retrieve the most similar sentence to the source language sentence to be translated. A hard retrieval set is composed of real sentence pairs;

[0012] Step 2: Perform M independent random sampling translations on the source language sentence to be translated. Each sampling generates a corresponding target language translation hypothesis. Combine the source language sentence to be translated with its corresponding target language translation hypothesis to form a soft search sentence pair. Combine all soft search sentence pairs into a soft search set.

[0013] Step 3: Merge the hard search set and the soft search set into a single example set, and format the example set into a prompt text according to the translation prompt template;

[0014] Step 4: Input the prompt text into a large language model with reasoning capabilities. After performing logical comparison, consistency check and local error correction on multiple examples in the prompt text, output the original answer text containing the reasoning process markers.

[0015] Step 5: Post-process and parse the original answer text to output the extracted final translation.

[0016] Furthermore, the semantic similarity function is as follows:

[0017] ;

[0018] in, Represents the semantic similarity function. This represents the source language sentence to be translated. This indicates the first in the real parallel corpus. A source language sentence, This represents a sentence vector encoding function, used to convert an input sentence into a real number vector. This represents the dot product operation. This indicates a modulo operation.

[0019] The values ​​of the number of examples K in the hard search set and the number of samples M in the soft search set are adjusted according to the resource richness of the target language pair to be translated. When the target language pair resources are scarcer, the value of M is larger. Both K and M are non-negative integers, and K and M cannot be zero at the same time.

[0020] Furthermore, the M independent random sampling translations of the source language sentence to be translated refer to: calling a pre-trained neural machine translation model, employing a Top-k sampling or kernel sampling strategy, and setting a temperature parameter. This is used to control the smoothness of the sampling probability distribution, and a maximum generation length threshold L is set. The length of the translation hypothesis generated in each sampling does not exceed the threshold L. The maximum generation length threshold L is adjusted according to the length and complexity of the source language sentence to be translated; the longer and more complex the source language sentence, the larger the value of L. L is a non-negative integer. The source language sentence to be translated is then processed... Each independent random sampling translation generates a corresponding target language translation hypothesis, resulting in a total of [number] hypotheses. Each translation hypothesis is paired with a source language sentence to be translated, forming a... One soft search query pair; among them; .

[0021] Furthermore, the M independent random sampling translations of the source language sentence to be translated refer to: when there is no directly trained neural machine translation model between the source language sentence and the target language, an intermediate language translation method is used to obtain translation hypotheses; specifically: the source language sentence to be translated is input into a source language to intermediate language neural machine translation model to obtain an intermediate translation; then the intermediate translation is input into an intermediate language to target language neural machine translation model, and the model is processed... After several independent random samplings, the following results were obtained. Each target language translation hypothesis is paired with a source language sentence to be translated, forming a... A pair of soft search terms, .

[0022] Furthermore, the M independent random sampling translations of the source language sentence to be translated refer to: when no available neural machine translation model exists, a rule-based machine translation system is used; under standard configuration, the source language sentence to be translated is translated once, generating a translation hypothesis; the experimental dictionary expansion option, transcription rule constraint option, morphological analyzer option, and unknown word guessing option of the rule-based machine translation system are adjusted; one option can be adjusted at a time or multiple options can be adjusted simultaneously, with the adjustment method gradually increasing the number of adjusted options from few to many. After each option adjustment, the source language sentence to be translated is translated once, generating a translation hypothesis, resulting in a total of Each translation hypothesis is paired with a source language sentence to be translated, forming a... One soft search query pair; among them; .

[0023] Furthermore, the prompt template includes: translation task instructions, examples from the hard search set, examples from the soft search set, and the source language sentence to be translated; the content is organized in the following order: first, the translation task instruction text is placed, then the examples from the hard search set are arranged in sequence, with "Real Example:" added before each hard search example as a prefix; then the examples from the soft search set are arranged in sequence, with "Reference Translation:" added before each soft search example as a prefix; finally, the source language sentence to be translated is placed; and "→?" is added at the end as the prompt end mark.

[0024] Furthermore, the inference process marker refers to placing the inference process text within a dedicated inference start marker. <think>"and the end of reasoning marker"< / think> The final translation is placed outside of these two marks.

[0025] Furthermore, the post-processing parsing refers to: searching for "" in the original answer <think> "and"< / think> Remove the two tags and all reasoning text between them, retaining the text outside the tags as the final extracted translation.

[0026] The low-resource machine translation system of the present invention includes the following functional modules:

[0027] The hard search module is used to calculate the similarity between the input source language sentence to be translated and each source language sentence in the real parallel corpus using a semantic similarity function. The similarity values ​​are sorted from high to low, and the top K sentence pairs are selected to form a hard search set.

[0028] The soft retrieval module is used to perform M independent random sampling translations on the input source language sentence to be translated, generate M translation hypotheses, and pair each translation hypothesis with the source language sentence to be translated to form a soft retrieval set;

[0029] The context suggestion building module is used to merge the input hard search set and soft search set into an enhanced context example set, and format the enhanced context example set into suggestion text according to the preset suggestion template;

[0030] The reasoning correction and generation module is used to feed the prompt text into a large language model with reasoning capabilities to generate an original answer containing tags of the reasoning process;

[0031] The post-processing parsing module is used to locate the original input answer, remove the inference process markers and the inference text between them, extract the final translation from the result, and output the final translation as the translation result.

[0032] Compared with the prior art, the present invention has the following advantages:

[0033] First, this invention uses a soft retrieval mechanism, which utilizes a neural machine translation model to generate pseudo-translation examples as a structured knowledge source. This overcomes the shortcomings of existing technologies that cannot construct effective retrieval contexts due to the lack of real parallel sentence pairs in extremely low-resource languages. This allows the invention to maintain the system's translation function even under extreme conditions where real examples are completely missing (K=0), by activating structural priors through the model's own generation capabilities.

[0034] Second, the present invention adopts a two-stage retrieval strategy that combines hard retrieval and soft retrieval, which organically combines high-confidence real examples with diverse but low-confidence pseudo examples as complementary information sources. This overcomes the shortcomings of existing technologies that rely on a single information source and are prone to introducing noise or failure when the corpus is sparse. This allows the present invention to construct robust enhanced context under limited data conditions, which significantly improves translation reliability.

[0035] Third, this invention introduces a large language model with reasoning capabilities. During the reasoning process, it performs logical comparison, consistency verification, and local error correction on multi-source examples. This overcomes the shortcomings of existing end-to-end translation methods, which cannot be undone once an error is generated and lack an interpretable error correction mechanism. This invention achieves a closed-loop optimization of "generation-reflection-correction", effectively alleviating the problems of semantic bias and word mismatch. Attached Figure Description

[0036] Figure 1 A flowchart illustrating an embodiment of the method of the present invention;

[0037] Figure 2 This is an algorithm flowchart of an embodiment of the method of the present invention. Detailed Implementation

[0038] The present invention will now be further described with reference to the accompanying drawings and embodiments.

[0039] Reference Figure 1 The implementation steps of the embodiments of the present invention will be described in further detail below.

[0040] Example 1: Spanish → Aragonese standard operating mode.

[0041] Application Scenario Description: The embodiments of this invention are applied to the document translation scenario within a digital cultural heritage preservation project. The project team needs to translate a batch of Aragonese folk tales (approximately 200 stories) written in Spanish into Aragonese. Aragonese is a very low-resource language, with fewer than 30,000 speakers worldwide, and only about 500 publicly available Spanish-Aragonese parallel corpora. The existing NLLB-200 model does not support Aragonese, and using GPT-4o for zero-shot translation often results in issues such as word mixing and article gender / number mismatches. The project team has obtained a small corpus containing 312 Spanish-Aragonese parallel sentence pairs and fine-tuned a Transformer-based neural machine translation model (approximately 60 million parameters) on this corpus. This model achieves a BLEU score of approximately 8.2 on the validation set, classifying it as a typical weak NMT model.

[0042] Specific implementation steps and parameter configuration.

[0043] Step 1: Input preparation.

[0044] Suppose the source language sentence to be translated is:

[0045] "El viejo pastor vive en una cabaña de piedra en la montaña."

[0046] The prepared real parallel sentence pair library Includes 312 Spanish-Aragonese sentence pairs. A trained NMT model. A Transformer-based architecture (6-layer encoder, 6-layer decoder, 8 attention heads, 512 hidden layer dimensions) was adopted and trained for 200 epochs on 312 parallel sentences. The Adam optimizer was used, and the learning rate was [missing information]. Use label smoothing ( The selected inference language model. For DeepSeek-R1 (671B parameters, using MoE architecture), the API interface is used for invocation, and the temperature parameter is set to 0.3. Hyperparameter settings: number of hard searches. Soft search sampling number .

[0047] Step 2: Hard search stage.

[0048] The LaBSE multilingual semantic encoder is used to translate the source language sentences. Parallel Sentence Pairing Library Encode all 312 source language sentences to obtain their respective sentence vector representations e. and Calculate semantic similarity:

[0049] ;

[0050] in, Represents the semantic similarity function. This represents the source language sentence to be translated. Indicates the first in the corpus A source language sentence, This represents the sentence vector encoding function. This represents the dot product operation. This indicates a modulo operation.

[0051] The three sentence pairs with the highest similarity constitute the hard retrieval set. ,in Represents the corresponding source language sentence The actual target language translation.

[0052] Step 3: Soft search stage.

[0053] The source language sentence to be translated Input the trained NMT model Kernel sampling is employed. Five independent samples were taken to generate five translation hypotheses. Temperature was set during decoding. The maximum generation length is set to 60 tokens, resulting in the translation hypothesis. These hypotheses are then paired with the source language sentences to be translated, forming a soft retrieval set. ,in For the first Translation hypotheses generated by subsampling.

[0054] Step 4: Contextual hints construction.

[0055] Will and Merged into a unified example set There are a total of 8 examples, formatted as prompt text according to a preset template. Hard retrieval examples are labeled "real examples", and soft retrieval examples are labeled "reference translations" to guide the large language model to distinguish information sources with different confidence levels.

[0056] Step 5: Inference and correction generation.

[0057] Input the above prompt into the DeepSeek-R1 model, and the model will... <think> ”、"< / think> "The system performs reasoning within the tags, compares real examples with candidate translations, identifies error patterns, extracts correct vocabulary and grammar rules, and generates the final translation."

[0058] Step Six: Post-processing analysis and result extraction.

[0059] Analyze the model output and locate " <think> ”、"< / think> "The reasoning process text between the markers is extracted, and the translation content outside the markers is obtained to obtain the final translation:"

[0060] "O viello pastor vive en una cabaña de piedra en a montaña."

[0061] Technical Results: The embodiments of this invention, through the synergy of hard and soft retrieval, successfully achieved accurate Spanish-Aragonese translation with extremely limited resources, using only 312 real parallel sentences. The soft retrieval mechanism generated five pseudo-examples rich in structural information through diversity sampling, supplementing the shortcomings of hard retrieval in sentence coverage; the large language model identified and corrected errors from multi-source information through reasoning, effectively solving the problems of lexical mismatch and grammatical errors, and improving translation quality.

[0062] Example 2: Chinese → Basque zero-sample extreme mode (K=0).

[0063] Application Scenario Description: This invention is applied to the localization of product information on a cross-border e-commerce platform. A cross-border e-commerce company plans to translate its Chinese product description pages into Basque (Euskara). Basque is an isolated language with approximately 750,000 speakers worldwide, making it a very low-resource language. Publicly available Chinese-Basque parallel corpora are virtually nonexistent. While NLLB-200 supports Basque, its translation quality in the Chinese-Basque direction is extremely low (ChrF2++ approximately 18.5). The company possesses an NMT model fine-tuned on a Spanish-Basque parallel corpus (approximately 2000 sentences) based on an mBART pre-trained model. This model performs reasonably well in the Spanish-Basque direction (BLEU ≈ 24.6), but has not been trained in the Chinese-Basque direction. Simultaneously, the company can utilize the Spanish-Basque translation module provided by the open-source rule-based machine translation system Apertium as an auxiliary tool.

[0064] Specific implementation steps and parameter configuration:

[0065] Step 1: Input preparation.

[0066] Suppose the source language sentence to be translated is:

[0067] "These wireless earbuds offer excellent sound quality and up to eight hours of battery life."

[0068] The prepared real parallel sentence pair library (Authentic examples in Chinese-Basque are completely missing), therefore a hard search set is required. Since a direct NMT model for Chinese-Basque does not exist, this embodiment of the invention employs an intermediate language bridging strategy: Spanish is used as the intermediate language, and the source language sentence to be translated is translated into Spanish using a general NMT model for Chinese-Spanish (OPUS-MT). Then, a fine-tuned NMT model for Spanish-Basque is used to generate Basque translation hypotheses. The selected inference large language model is GPT-4o, and hyperparameters are set accordingly. , .

[0069] Step 2: Hard search stage.

[0070] because Hard search set This stage will be skipped. Among them, This is a database of authentic parallel sentences in Chinese and Basque.

[0071] Step 3: Soft retrieval stage (intermediate language mode).

[0072] First, the Chinese source sentence is... (OPUS-MT) translates to Spanish middle sentence Then Input Spanish → Basque NMT Model (mBART-50 fine-tuning model), using Top-k sampling ( Eight independent samples were taken, and the temperature was... Eight Basque translation hypotheses were generated. This constitutes a soft search set. .in, A general NMT model representing Chinese to Spanish. This represents a fine-tuned NMT model from Spanish to Basque. The source Chinese sentence is to be translated. For the first A Basque translation hypothesis.

[0073] Steps four through six: Contextual hint construction, inference correction generation, post-processing parsing, and result extraction.

[0074] The prompts were constructed using the same method as in Example 1, GPT-4o was input for inference correction, the output results were parsed, and the translation was extracted. The final translation was obtained after parsing the model's output.

[0075] "Aurikular inalambriko honek soinu-kalitate bikaina eta zortzi ordukobateria iraupena du.".

[0076] Technical effect: The embodiments of the present invention demonstrate that the present invention can be implemented without any real-world examples ( It remains effective even under extreme conditions. The soft retrieval mechanism constructs eight pseudo-examples rich in structural information through intermediate language bridging and diversity sampling; the large language model achieves high-confidence translation decisions by analyzing the consistency patterns of multiple candidates (such as most translation assumptions using "aurikularinalambriko honek" as the subject phrase), overcoming the deficiency of traditional methods that cannot be started in zero-shot scenarios.

[0077] Example 3: English → Aramaic Lightweight Configuration Mode (M=3).

[0078] Application Scenario Description: The embodiments of this invention are applied to an education equity project in Catalonia. Aranese is one of the official languages ​​of the Valley of Aranese in Catalonia, Spain, with approximately 5,000 speakers. The local education department needs to translate a batch of English science textbooks into Aranese. The project team has constructed a corpus containing 487 parallel English-Aranese sentence pairs and fine-tuned an NMT model (approximately 210 million parameters) based on the Transformer-large architecture on this corpus. The model achieves a BLEU score of approximately 12.7 on the test set.

[0079] Specific implementation steps and parameter configuration:

[0080] Step 1: Input preparation.

[0081] Suppose the source language sentence to be translated is:

[0082] "Water freezes at zero degrees Celsius and boils at one hundreddegrees."

[0083] The prepared real parallel sentence pair library It contains 487 English-Arana parallel sentence pairs. The trained NMT model... The Transformer-large architecture was used, and the model was trained for 300 rounds on a corpus of 487 parallel sentences. The chosen large-scale inference language model was DeepSeek-R1-Distill-Qwen-7B (a lightweight inference model compressed using distillation techniques), deployed on a local server (single NVIDIA A10G GPU, 24GB VRAM), employing the vLLM inference framework, with the temperature parameter set to 0.2. Hyperparameters were set as follows: , .

[0084] Step 2: Hard search stage.

[0085] Using Sentence-BERT to translate source language sentences With corpus All 487 source language sentences were encoded, semantic similarity was calculated, and the 5 sentence pairs with the highest similarity were selected to form a... .in, The source English sentence is to be translated. This is a library of parallel sentences in English and Aramaic.

[0086] Step 3: Soft search stage.

[0087] The source language sentence to be translated Input the trained NMT model Top-k sampling is used. Three independent samples were taken (temperature). ), resulting in 3 translation hypotheses, constituting .in, For an English-Arab NMT model based on the Transformer-large architecture, For the first Translation hypotheses generated by subsampling.

[0088] Steps four through six: Contextual hint construction, inference correction generation, post-processing parsing, and result extraction.

[0089] Following the same method as in Example 1, a prompting and input lightweight inference model are constructed, inference correction is performed, and the output results are parsed. The final translation is obtained after parsing the model's output.

[0090] "La'aigua se gèla a zèro grads e bull a cent grads.".

[0091] Technical Effects: The embodiments of this invention verify its flexible adaptability under different computing resource conditions. By replacing the full version of DeepSeek-R1 (671 billion parameters) with DeepSeek-R1-Distill-Qwen-7B (7 billion parameters), the deployment threshold is significantly reduced while maintaining inference capabilities. The soft search configuration is different from that in Implementation Example 1. This reduced computational costs by 40%, proving the effectiveness of hyperparameters. It allows for a flexible balance between quality and efficiency depending on the application scenario, making low-resource language translation technology universally accessible.

[0092] Example 4: French → Aramaic rule MT substitution pattern.

[0093] Application scenario description.

[0094] The embodiments of this invention are applied to scenarios within the open-source machine translation community. Apertium is an open-source rule-based machine translation platform that supports various low-resource language pairs (e.g., French → Aramaic), including Aramaic. However, its translation quality, based on hand-written dictionaries and transcription rules, is limited by rule coverage. The Apertium community aims to improve the translation quality of French → Aramaic without retraining an NMT model. The embodiments of this invention demonstrate an alternative solution that allows the invention to run without an NMT model.

[0095] Specific implementation steps and parameter configuration.

[0096] Step 1: Input preparation.

[0097] Suppose the source language sentence to be translated is:

[0098] "Le chat noir dort sur le toit de la vieille ferme."

[0099] The prepared real parallel sentence pair library It contains 215 French-Aramaic parallel sentence pairs. This embodiment of the invention does not use an NMT model for soft retrieval; instead, it uses five different configurations of the Apertium rule-based translation system (standard version, enabling experimental dictionary expansion, relaxing transcription rule constraints, using different morphological analyzers, and enabling the unknown word guessing module) to generate multiple translation hypotheses. The selected inference large language model is Claude-3 Opus, with the temperature parameter set to 0.2. Hyperparameter settings: , (5 Apertium configurations).

[0100] Step 2: Hard search stage.

[0101] Using the cross-language sentence encoder BERTScore to translate the source language sentence With corpus The semantic similarity of sentences in the source language is calculated, and the three sentence pairs with the highest similarity are selected to form a semantic similarity score. .in, The source language sentence is French, which needs to be translated. This is a French-Aryan parallel sentence pair library.

[0102] Step 3: Soft search stage (Apertium alternative).

[0103] The source language sentence to be translated Five translation hypotheses were obtained by inputting five different configurations of the Apertium system (standard version, experimental dictionary expansion enabled, relaxed transcription rule constraints, different morphological analyzers, and unknown word guessing module enabled). ,constitute .in, Indicates the first Translation hypotheses output by the Apertium rule-based translation system with this configuration.

[0104] Steps four through six: Contextual hint construction, inference correction generation, post-processing parsing, and result extraction.

[0105] The prompts were constructed using the same method as in Example 1, and the Claude-3 Opus was used for inference correction and output parsing. The final translation was obtained after parsing the model's output.

[0106] "Lo gat negre dorm sus lo teit de la vièlha bòrda."

[0107] Technical Effects: The embodiments of this invention demonstrate that the soft retrieval mechanism is not limited to NMT models; various configurations of the Apertium rule system can also serve as soft retrieval sources. The large language model obtains the basic sentence body through hard retrieval and key lexical comparisons (e.g., "vieille" → "vièlha", "ferme" → "bòrda") through soft retrieval, then combines the two to achieve error correction. This approach enables the invention to cover language pairs that even weak NMT models cannot train, and the translation process can be initiated with just a configurable rule-based NMT system, demonstrating significant value for open-source community applications.

[0108] The effectiveness of this invention can be further demonstrated through the following simulation.

[0109] 1. Simulation experimental conditions.

[0110] The simulation environment for this invention is an Ubuntu 24.04 operating system with 64GB of RAM and two NVIDIA RTX 3090 graphics cards (24GB VRAM each). In local experiments, all experiments related to the 14B model were run on dual cards, while all experiments related to models <14B were run on a single card. Furthermore, experiments with the DeepSeek-R1-Distill-Qwen-32B model were completed using the API provided by the Siliconflow platform (https: / / siliconflow.cn). All methods were evaluated using SacreBLEU within a unified framework, specifically employing three metrics: ChrF2++, spBLEU-101, and spBLEU-200.

[0111] Table 1. Performance overview of different methods on Spa→Arg, Spa→Arn, and Spa→Ast tasks.

[0112]

[0113] 2. Simulation content and result analysis.

[0114] The simulation experiments of this invention consist of three parts, specifically including the following three very low-resource translation tasks: Spanish to Aragon (Spa→Arg), Spanish to Aran (Spa→Arn), and Spanish to Asturian (Spa→Ast). The data used in the simulation experiments of this invention is the FLORES+ multilingual translation benchmark, which includes a validation set (dev) with 997 sentence pairs and a development test set (devtest) with 1012 sentence pairs.

[0115] Currently, NLLB-200 (No Language Left Behind) is the only multilingual neural machine translation model that supports up to 200 languages. It not only has the widest coverage but also significantly outperforms other existing systems in translation quality, thus serving as the primary baseline for comparison. Besides NLLB-200, given that the method proposed in this invention is based on a large language model with reasoning capabilities, the comparison method also includes a reasoning large language model and its corresponding basic large language model. The basic large language model (Vanilla LLM) refers to a basic large language model without a reasoning mechanism. Considering that the DeepSeek-R1 reasoning model used in the simulation experiments of this invention is based on the Qwen2.5 series, two typical-scale models from the Qwen2.5 series, Qwen2.5-1.5B and Qwen2.5-7B, are selected as Vanilla LLM comparisons. The reasoning large language model (Reasoning LLM) refers to a large language model with structured reasoning capabilities. The simulation experiments of this invention use DeepSeek-R1 as the core and build upon it a low-resource machine translation method based on soft and hard retrieval and reasoning correction. Specific models include DeepSeek-R1-Distill-Qwen 1.5B, 7B, 8B, 14B, and 32B.

[0116] The simulation results are shown in Table 1. In the table, the results of each task correspond to the three indicators ChrF2++, spBLEU-101, and spBLEU-200, respectively. The bolded values ​​represent the best results in the corresponding columns.

[0117] Analysis of simulation experiment results.

[0118] Multilingual Neural Machine Translation Model: Under the experimental settings used in the simulation experiments of this invention, the embodiments of this invention reproduced the performance of NLLB-200 on the Spanish → Asturian task, with a ChrF2++ score of 46.29, which is highly consistent with the official reported value (46.2), indicating that the simulation experiments of this invention have reliability and fairness in implementing the baseline model. Further observation revealed that NLLB-200 performed significantly better on the Spanish → Asturian task (ChrF2++=46.29) than the other two directions, while its performance dropped sharply on the Spanish → Aragonese and Spanish → Aranese tasks (ChrF2++ were 14.27 and 12.21, respectively; spBLEU-101 was only 2.61 and 1.66). This suggests that the training corpus of NLLB-200 may contain a certain amount of parallel Asturian data, thus enabling it to generalize to some extent in this direction; while Aragonese and Aran languages ​​suffer from a severe lack of training resources, making it difficult for the model to effectively model them, highlighting the limitations of current large-scale multilingual neural machine translation models in covering long-tail languages.

[0119] Basic Large Language Model: Compared to dedicated neural machine translation systems, the basic large language model, while possessing powerful general language understanding capabilities, exhibits inconsistent performance in low-resource translation tasks. Specifically, in the Spanish-Aragonese and Spanish-Aranaese tasks, the large language model significantly outperforms NLLB-200 with only a few examples. We believe this improvement stems from the massive amount of multilingual monolingual texts it encounters during pre-training, especially considering that Aragonese, Aranaese, and Asturian belong to the Romance language family and share a large number of morphological, syntactic, and lexical features with multi-resource languages ​​such as Spanish, French, and Italian. Therefore, when learning multi-resource languages, the model indirectly learns the latent patterns of low-resource related languages, thus achieving knowledge transfer in translation. However, in the Spanish-Asturian task, the basic large language model does not show a significant advantage. This may be because a considerable amount of parallel corpus is available in this direction, allowing data-driven neural machine translation models to learn specific translation mappings more focusedly, thus showing an advantage in task adaptability.

[0120] Inference-based large language models: In comparison, the method proposed in this invention achieves state-of-the-art performance across all tasks. Particularly noteworthy is the significant lead achieved by DeepSeek-R1-32B in the Spanish-Aragonese translation task, with ChrF2++, 46.60 and spBLEU-101 / 200, 18.12 / 19.06 scores; it also achieves the highest spBLEU score in the Spanish-Aragonese translation task (spBLEU-101 / 200, 3.59 / 3.90); and similarly achieves state-of-the-art results in the Spanish-Asturian translation task (ChrF2++, 47.95, spBLEU-101 / 200, 24.47 / 23.02). This result fully validates that by combining soft and hard retrieval with inference correction, large language models can outperform dedicated neural machine translation systems in extremely low-resource translation scenarios. Furthermore, the method proposed in this invention comprehensively outperforms multilingual neural machine translation models and basic large language models across all tasks and metrics. Even on the Spanish-Asturian task, where it performed best in NLLB-200, it achieved further improvements. Furthermore, even the smallest 1.5B version significantly outperformed the basic large language models of the same size. This indicates that the performance gains primarily stem from the soft and hard retrieval and inference correction mechanisms proposed in this invention: when both reference examples and initial translation hypotheses are provided simultaneously, the model can fuse multi-source information and generate more accurate and fluent translations through inference correction. Additionally, the parameter size exhibits a positive synergistic effect on the inference large language model. DeepSeek-R1-32B, as the largest model, achieved the best overall results, demonstrating that the increased model capacity, after introducing structured inference capabilities, effectively translates into stronger task-solving ability, thus avoiding the scale trap seen in basic large language models. Finally, comparing the performance of DeepSeek-R1 and Qwen2.5 at 1.5B and 7B scales shows that on the Spanish-Asturian task, increasing the number of parameters not only did not lead to overfitting but also resulted in performance improvements. This further confirms the effectiveness of the soft and hard retrieval and inference correction mechanisms. In particular, the local relevant context examples provided by soft retrieval provide the model with strong semantic constraints and domain priors, effectively alleviating the generalization problem in low-resource scenarios.< / think>

Claims

1. A low-resource machine translation method based on soft and hard retrieval and inference correction, characterized in that, The steps of this translation method include: Step 1: Use a semantic similarity function to retrieve the most similar sentence to the source language sentence to be translated. A hard retrieval set is composed of real sentence pairs; Step 2: Perform M independent random sampling translations on the source language sentence to be translated, generate the target language translation hypothesis corresponding to each sampling, and form a soft search sentence pair with the source language sentence to be translated and its corresponding target language translation hypothesis. Then, form a soft search set with all the soft search sentence pairs. Step 3: Merge the hard search set and the soft search set into a single example set, and format the example set into a prompt text according to the translation prompt template; Step 4: Input the prompt text into a large language model with reasoning capabilities. After performing logical comparison, consistency check and local error correction on multiple examples in the prompt text, output the original answer text containing the reasoning process markers. Step 5: Post-process and parse the original answer text to output the extracted final translation.

2. The low-resource machine translation method according to claim 1, characterized in that, The semantic similarity function mentioned in step 1 is as follows: ； in, Represents the semantic similarity function. This represents the source language sentence to be translated. This indicates the first in the real parallel corpus. A source language sentence, This represents a multilingual semantic encoder, used to convert an input sentence into a real number vector. This represents the dot product operation. This indicates a modulo operation.

3. The low-resource machine translation method according to claim 2, characterized in that, The values ​​of the number of examples K in the hard search set and the number of samples M in the soft search set are adjusted according to the resource richness of the target language pair to be translated. When the target language pair resources are scarcer, the value of M is larger. Both K and M are non-negative integers, and K and M cannot be zero at the same time.

4. The low-resource machine translation method according to claim 3, characterized in that, Step 2, which involves performing M independent random sampling translations on the source language sentence to be translated, refers to: calling a pre-trained neural machine translation model, employing a Top-k sampling or kernel sampling strategy, and setting a temperature parameter. This is used to control the smoothness of the sampling probability distribution, and a maximum generation length threshold L is set. The length of the translation hypothesis generated in each sampling does not exceed the threshold L. The maximum generation length threshold L is adjusted according to the length and complexity of the source language sentence to be translated; the longer and more complex the source language sentence, the larger the value of L. L is a non-negative integer. The source language sentence to be translated is then processed... Each independent random sampling translation generates a corresponding target language translation hypothesis, resulting in a total of [number] hypotheses. Each translation hypothesis is paired with a source language sentence to be translated, forming a... One soft search query pair; among them; .

5. The low-resource machine translation method according to claim 3, characterized in that, Step 2, which involves performing M independent random sampling translations of the source language sentence to be translated, refers to the following: when there is no directly trained neural machine translation model between the source language sentence and the target language, an intermediate language translation method is used to obtain translation hypotheses. Specifically, the source language sentence to be translated is input into a source-to-intermediate language neural machine translation model to obtain an intermediate translation; then, this intermediate translation is input into an intermediate-to-target language neural machine translation model, and the model is processed... After several independent random samplings, the following results were obtained. Each target language translation hypothesis is paired with a source language sentence to be translated, forming a... A pair of soft search terms, .

6. The low-resource machine translation method according to claim 3, characterized in that, Step 2, which involves performing M independent random sampling translations of the source language sentence to be translated, refers to the following: When no available neural machine translation model exists, a rule-based machine translation system is used. Under standard configuration, the source language sentence to be translated is translated once, generating a translation hypothesis. The following options are adjusted in the rule-based machine translation system: enabling experimental dictionary expansion, enabling transcription rule constraints, enabling morphological analyzer, and enabling unknown word guessing. One option can be adjusted at a time, or multiple options can be adjusted simultaneously, gradually increasing the number of adjusted options from few to many. After each option adjustment, the source language sentence to be translated is translated once, generating a translation hypothesis. A total of [number] hypotheses are obtained. Each target language translation hypothesis is formulated; all translation hypotheses are paired with sentences in the source language to be translated, forming... A pair of soft search terms, .

7. The low-resource machine translation method according to claim 1, characterized in that, The prompt template mentioned in step 3 includes: translation task instructions, examples from the hard search set, examples from the soft search set, and the source language sentence to be translated; Organize the content in the following order: First, place the translation task instruction text, then arrange the examples in the hard search set in sequence, adding "Real Example:" as a prefix to each hard search example; next, arrange the examples in the soft search set in sequence, adding "Reference Translation:" as a prefix to each soft search example; finally, place the source language sentence to be translated; and add "→?" at the end as a prompt end mark.

8. The low-resource machine translation method according to claim 1, characterized in that, The reasoning process marker mentioned in step 4 refers to placing the reasoning process text in a dedicated reasoning start marker. <think> "and the end of reasoning marker"< / think> The final translation is placed outside of these two marks.

9. The low-resource machine translation method according to claim 1, characterized in that, The post-processing parsing mentioned in step 5 refers to: searching for "" in the original answer <think> "and"< / think> Remove the two tags and all reasoning text between them, retaining the text outside the tags as the final extracted translation.

10. A low-resource machine translation system based on soft and hard retrieval and inference correction, characterized in that, Based on the low-resource machine translation method of any one of claims 1-9, the system includes the following functional modules: The hard search module is used to calculate the similarity between the input source language sentence to be translated and each source language sentence in the real parallel corpus using a semantic similarity function. The similarity values ​​are sorted from high to low, and the top K sentence pairs are selected to form a hard search set. The soft retrieval module is used to perform M independent random sampling translations on the input source language sentence to be translated, generate M translation hypotheses, and pair each translation hypothesis with the source language sentence to form a soft retrieval set; The context suggestion building module is used to merge the input hard search set and soft search set into an enhanced context example set, and format the enhanced context example set into suggestion text according to the preset suggestion template; The collaborative reasoning generation module is used to feed the prompt text into a large language model with reasoning capabilities to generate an original answer containing tags of the reasoning process; The post-processing parsing module is used to locate the original input answer, remove the inference process markers and the inference text between them, extract the final translation from the result, and output the final translation as the translation result.

Images