Translation-based multi-language reasoning method and device, equipment, storage medium and program product

Abstract

The invention provides a translation-based multi-language reasoning method and device, equipment, a storage medium and a program product, and relates to the technical field of translation reasoning, the method comprises the steps that a large language model is acquired, and the large language model comprises a cross-language reasoning layer, an autonomous translation layer and a target language reasoning layer; based on a language prompt, processing a target English question through the cross-language reasoning layer to generate a first target language answer; translating the target English question through the autonomous translation layer to generate a self-translation question; reasoning the self-translation question through the target language reasoning layer to generate a second target language answer; and based on the first target language answer and the second target language answer, training the large language model to obtain a target large language model, and generating a multi-language reasoning result based on the target large language model. The multi-language reasoning capability of the large language model can be improved.

Description

Technical Field

[0001] This invention relates to the field of translation reasoning technology, and in particular to a multilingual reasoning method, apparatus, device, storage medium, and program product based on translation. Background Technology

[0002] With the rapid development of large language models (LLM), reasoning models with long thought chain (LCT) capabilities (such as OpenAI o1[1], DeepSeek-R1[2], etc.) have made significant breakthroughs in complex tasks such as mathematics, coding, and logical reasoning. However, due to the dominance of English in the pre-training data, these models often lag significantly behind English in reasoning ability when dealing with non-English (especially low-resource languages) tasks, and are prone to "language inconsistency" (i.e., the input is non-English, but the model mainly uses English to think and answer, resulting in performance degradation), which will seriously affect the multilingual reasoning ability of large language models. Summary of the Invention

[0003] This invention provides a translation-based multilingual reasoning method, apparatus, device, storage medium, and program product to address the shortcomings of existing technologies where the lack of low-resource language training data affects the multilingual reasoning capabilities of large language models.

[0004] This invention provides a translation-based multilingual reasoning method, comprising the following steps: Obtain a large language model, wherein the large language model includes a cross-language reasoning layer, an autonomous translation layer, and a target language reasoning layer; Based on language prompts, the target English question is processed through the cross-language reasoning layer to generate a first target language answer; The target English question is translated using the autonomous translation layer to generate a self-translated question; The self-translation question is reasoned through the target language reasoning layer to generate a second target language answer; Based on the first target language answer and the second target language answer, the large language model is trained to obtain a target large language model, and multilingual reasoning results are generated based on the target large language model.

[0005] According to a translation-based multilingual reasoning method provided by the present invention, before processing the target English question through the cross-language reasoning layer to generate a first target language answer, the method further includes: Initial English questions are randomly selected from the English reasoning dataset; Based on the language prompts, the cross-language reasoning layer samples and answers the initial English question, and outputs multiple initial answers corresponding to the initial English question. The language prompts are used to guide the cross-language reasoning layer to answer the English question using the target language. Based on the standard answer, the accuracy rate of the answer to the initial English question is calculated according to the multiple initial answers, and the initial English questions with an accuracy rate greater than a predefined threshold are identified as target English questions.

[0006] According to the present invention, a translation-based multilingual reasoning method is provided, wherein training a large language model based on the first target language answer and the second target language answer to obtain a target large language model includes: Based on the standard answer, the first stage reward value is calculated according to the first target language answer, and the second stage reward value and the third stage reward value are calculated according to the second target language answer; Calculate the comprehensive reward value based on the first stage reward value, the second stage reward value, and the third stage reward value; Based on the comprehensive reward value, the large language model is trained to obtain the target large language model.

[0007] According to a translation-based multilingual reasoning method provided by the present invention, the step of calculating a first-stage reward value based on a standard answer in a first target language, and calculating a second-stage reward value and a third-stage reward value based on a second target language answer, includes: Based on the standard answer, the first stage reward value is calculated according to the first format reward, the first language consistency reward, the first accuracy reward, and the first duplication reward of the first target language answer; Based on the standard answer, the second-stage reward value is calculated according to the accuracy rate of the answer in the second target language; Based on the standard answer, the second stage reward value is calculated according to the second format reward, second language consistency reward, second accuracy reward, and second repetition reward of the second target language answer, and then the third stage reward value is calculated.

[0008] According to the present invention, a translation-based multilingual reasoning method is provided, wherein training the large language model based on the comprehensive reward value to obtain a target large language model includes: Based on the comprehensive reward value, the large language model is trained using a group relative policy optimization algorithm to obtain the target large language model.

[0009] According to the present invention, a translation-based multilingual reasoning method is provided, wherein the target large language model is obtained by training the large language model through a group relative policy optimization algorithm based on the comprehensive reward value, comprising: The target English question is sampled multiple times using the large language model to generate multiple different output groups, where each output group includes a first target language answer and a second target language answer. Calculate the combined reward value for the first target language answer and the second target language answer in each output group; Based on the combined reward value of all output groups under the same target English question, calculate the relative advantage score of each output group; The strategy gradient is calculated based on the relative advantage score, and the model parameters of the large language model are iteratively updated based on the strategy gradient until the model converges, thus obtaining the target large language model.

[0010] This invention also provides a translation-based multilingual reasoning method, comprising the following modules: The acquisition module is used to acquire a large language model, wherein the large language model includes a cross-language reasoning layer, an autonomous translation layer, and a target language reasoning layer; The generation module is used to process the target English question based on language prompts through the cross-language reasoning layer to generate a first target language answer; The generation module is also used to translate the target English question through the autonomous translation layer to generate a self-translated question; The generation module is also used to reason about the self-translation question through the target language reasoning layer to generate a second target language answer; The training module is used to train the large language model based on the first target language answer and the second target language answer to obtain a target large language model, so as to generate multilingual reasoning results based on the target large language model.

[0011] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the translation-based multilingual reasoning method described above.

[0012] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the translation-based multilingual reasoning method as described above.

[0013] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the translation-based multilingual reasoning method as described above.

[0014] This invention provides a translation-based multilingual reasoning method, apparatus, device, storage medium, and program product. It involves acquiring a large language model, which includes a cross-language reasoning layer, a self-translating layer, and a target language reasoning layer. Based on language prompts, the cross-language reasoning layer processes a target English question to generate a first target language answer. The self-translating layer translates the target English question to generate a self-translated question. The target language reasoning layer infers the self-translated question to generate a second target language answer. Based on the first and second target language answers, the large language model is trained to obtain a target large language model, which is then used to generate multilingual reasoning results. This invention addresses the limitation of large language models' multilingual reasoning capabilities caused by a lack of low-resource language training data. Compared to existing technologies, it does not rely on large-scale low-resource language annotation data. Through a dual-path collaborative reasoning architecture consisting of a cross-language reasoning layer, an autonomous translation layer, and a target language reasoning layer, it leverages cross-language knowledge transfer to ensure the first target language answer generated by the cross-language reasoning layer accurately matches the core logic of the target English question, guaranteeing semantic consistency. Simultaneously, the autonomous translation layer accurately transforms the target English question into a self-translated question. Finally, the target language reasoning layer generates a second target language answer that conforms to the usage habits of the target language, ensuring idiomatic expression. The complementary supervisory signals provided by the two types of answers generated through the dual paths further enhance... The learning and training process provides rich and effective optimization criteria, enabling the model to continuously reduce the deviation between the two-path answers during the iteration process. This avoids the semantic loss that easily occurs in the "direct translation followed by reasoning" of existing technologies, which leads to reasoning deviating from the core of the original problem. It also solves the problem of stiff expression in "single cross-language reasoning". At the same time, it significantly improves the model's adaptability and generalization ability to low-resource languages, even niche languages ​​and dialect expressions. There is no need to build a separate labeled dataset for each low-resource language, which greatly reduces the training cost and deployment threshold of multilingual models. Ultimately, the multilingual reasoning results generated by the target large language model take into account both logical accuracy and linguistic practicality, effectively breaking through the limitation of multilingual reasoning ability caused by the lack of training data for low-resource languages. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0016] Figure 1 This is one of the flowcharts of the translation-based multilingual reasoning method provided by the present invention.

[0017] Figure 2 This is a schematic diagram of model training for the translation-based multilingual reasoning method provided by the present invention.

[0018] Figure 3 This is the second flowchart of the translation-based multilingual reasoning method provided by the present invention.

[0019] Figure 4 This is a schematic diagram of the structure of the translation-based multilingual reasoning device provided by the present invention.

[0020] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0022] The following is combined with Figure 1 and Figure 3 The present invention describes a translation-based multilingual reasoning method applicable to any translation-based multilingual reasoning. The execution subject of this method can be an electronic device or a translation-based multilingual reasoning device installed in the electronic device. The translation-based multilingual reasoning device can be implemented by software, hardware, or a combination of both.

[0023] Figure 1 This is one of the flowcharts illustrating the translation-based multilingual reasoning method provided by this invention, such as... Figure 1 As shown, the method includes the following: Step 101: Obtain a large language model, wherein the large language model includes a cross-language reasoning layer, an autonomous translation layer, and a target language reasoning layer.

[0024] It should be noted that the cross-language reasoning layer is mainly used to directly perform cross-language reasoning operations based on the target English questions selected from the English reasoning dataset and the preset target language guiding prefixes, without relying on external translation tools, to generate a first target language answer containing a complete thought process and a final answer; the autonomous translation layer is mainly used to receive the target English questions selected by the cross-language reasoning layer and convert them into self-translated questions in the target language without external translation assistance, and the translation quality is determined by the closed-loop reasoning results of the subsequent target language reasoning layer; the target language reasoning layer is mainly used to receive the self-translated questions output by the autonomous translation layer and generate a second target language answer containing a thought process and a final answer.

[0025] Step 102: Based on language prompts, process the target English question through the cross-language reasoning layer to generate a first target language answer; It should be noted that the language prompt is a preset target language guiding prefix. Its core function is to enforce the output language of the cross-language reasoning layer, avoiding the generation of non-target language or mixed language content. Based on this language prompt, the cross-language reasoning layer directly performs end-to-end cross-language reasoning operations on the target English question without relying on any external translation tools. It deeply integrates the semantic understanding of the English question with the reasoning expression in the target language, ultimately generating a first target language answer that includes a complete thought process and a clear final answer. The target English question refers to a high-quality training sample selected from the English reasoning dataset by the cross-language reasoning model.

[0026] Understandably, for the cross-language inference stage, question screening can be performed first. Not all English questions are suitable for multilingual training of the current model. The cross-language inference layer can be used to verify whether the model can correctly answer in the target language. This process involves selecting high-quality samples from which the model demonstrates potential problem-solving capabilities in cross-linguistic contexts. This selection mechanism essentially constructs a dynamic "learning curriculum," preventing the model from being forced to generate multilingual data for problems it cannot understand or reason about, thus avoiding the illusion of success and ensuring the quality of training data and the stability of the learning process in subsequent stages. Secondly, it stimulates the model's multilingual capabilities, enabling it to answer questions in the target language. The process also improved the model's target language generation capabilities. Finally, a selection process was conducted, and the model was validated against a standard answer. The correctness of the answer. Only answers with an accuracy rate greater than a predefined threshold are considered correct. These will be retained for future optimization to ensure the accuracy of subsequent rewards.

[0027] Step 103: Translate the target English question through the autonomous translation layer to generate a self-translated question; It should be noted that the autonomous translation layer is mainly used to guide the large language model to translate the target English question into the target language, generating the autonomous translation question. The innovative design of this stage lies in the fact that no external translator or reference translation is used to evaluate translation quality, nor is a translation reward given immediately. The quality of the translation will be entirely verified by the inference effect of the subsequent target language inference layer. In addition, the translation task will also guide the model to align the target language with English, thereby improving the model's multilingual capabilities.

[0028] Understandably, the input to the autonomous translation layer is a high-quality target English question filtered by the cross-language inference layer. Without any external translation tools, reference translations, or human proofreading assistance, this layer autonomously completes the semantic conversion between English and the target language, accurately translating the target English question into a self-translated question that conforms to the expression habits of the target language and is adapted to the model's own inference logic. This translation process does not have an independent real-time quality assessment and reward mechanism. Instead, the judgment of the translation quality and the generation of the corresponding third-stage reward value are completely left to the closed-loop verification of the inference results of the subsequent target language inference layer. This not only achieves semantic alignment between English and the target language, but also provides core input samples for the target language inference layer. At the same time, it builds the foundation for the linkage between translation and inference tasks, allowing the optimization of translation behavior to be iteratively improved based on the feedback of inference results.

[0029] Step 104: Reason about the self-translation problem through the target language reasoning layer to generate a second target language answer; It should be noted that the target language inference layer, based on the inference logic, expression norms, and semantic understanding capabilities of the target language, can perform end-to-end target language inference operations on the self-translation problem without the assistance of external inference tools. This generates a second target language answer that includes a complete thought chain derivation process and a clear final answer. This answer is not only a direct output of the model's target language inference capability but also the core basis for calculating the second-stage reward value. Furthermore, its equivalence matching result with the corresponding standard answer is used to evaluate the translation quality of the autonomous translation layer and generate the third-stage reward value. This constructs a closed-loop optimization mechanism of "translation-inference-feedback translation," providing key data support for the subsequent training of the Group Relative Policy Optimization (GRPO) algorithm and promoting the synergistic improvement of the model's target language inference capability and autonomous translation capability.

[0030] Step 105: Based on the first target language answer and the second target language answer, train the large language model to obtain a target large language model, and generate multilingual reasoning results based on the target large language model.

[0031] It should be noted that the Large Language Model can be trained using the Group Relative Policy Optimization (GRPO) algorithm. Specifically, the first-stage reward value can be calculated based on four dimensions: format compliance, target language purity, answer accuracy, and content non-repetition of the first target language answer. Then, the second-stage reward value is calculated based on the accuracy of the second target language answer. Simultaneously, the third-stage reward value is calculated based on the same four dimensions. Finally, these reward values ​​are combined, and the GRPO algorithm is used for policy optimization to adjust the model parameters of the Large Language Model to maximize the total reward, ultimately training a performance-enhanced Large Language Model.

[0032] Understandably, the target language model refers to a large language model that has undergone multi-stage reward-guided GRPO reinforcement learning training, achieving collaborative optimization of the cross-language reasoning layer, autonomous translation layer, and target language reasoning layer, thus possessing stable cross-language reasoning capabilities. The multilingual reasoning result, on the other hand, is the output of this model after processing an input English question without external translation or reasoning tools, through a complete chain of internal cross-language semantic understanding, autonomous translation, and target language reasoning. This type of result must strictly meet the requirements of format compliance (conforming to preset expression norms), linguistic purity (using the target language throughout without mixing), logical consistency (coherent and uninterrupted reasoning), accuracy (equivalent to the standard answer), and no repetition or redundancy. It can flexibly output reasoning content in different target languages ​​such as Chinese and Japanese according to actual task needs, including both a clear and easy-to-understand reasoning process and a clearly stated final conclusion, directly adapting to the reasoning needs of various scenarios such as educational Q&A, technical document analysis, and cross-language consultation.

[0033] In specific implementations, such as Figure 2 As shown, a three-stage closed-loop training mechanism is adopted for training large language models. This mechanism achieves model self-optimization through iterative data flow and quality screening: In the first stage, an English question is input, and the model directly generates a target language answer. Accuracy is then used to screen the data based on the target language standard answer, retaining only high-quality data that meets the accuracy criteria. In the second stage, English questions with appropriate difficulty are extracted from the screened high-accuracy data. These are then translated into target language questions using a translation module. The translation quality is evaluated based on the correctness of the target language answer from the first stage, ensuring that the translated question is semantically consistent with the original question and matches its difficulty. In the third stage, the translated target language question is input into the model, generating another target language answer to enrich the target language reasoning data. Finally, all data generated in the three stages is used for model training, forming a complete closed loop of "input → screening → translation → re-input → full training." This ensures the quality and diversity of training data without relying on additional low-resource language annotation data, effectively improving the model's reasoning ability in low-resource language scenarios.

[0034] The multilingual reasoning method based on translation provided in this invention involves acquiring a large language model, which includes a cross-language reasoning layer, a self-translating layer, and a target language reasoning layer. Based on language prompts, the cross-language reasoning layer processes a target English question to generate a first target language answer. The self-translating layer translates the target English question to generate a self-translated question. The target language reasoning layer infers the self-translated question to generate a second target language answer. Based on the first and second target language answers, the large language model is trained to obtain a target large language model, which is then used to generate multilingual reasoning results. This invention addresses the limitation of large language models' multilingual reasoning capabilities caused by a lack of low-resource language training data. Compared to existing technologies, it does not rely on large-scale low-resource language annotation data. Through a dual-path collaborative reasoning architecture consisting of a cross-language reasoning layer, an autonomous translation layer, and a target language reasoning layer, it leverages cross-language knowledge transfer to ensure the first target language answer generated by the cross-language reasoning layer accurately matches the core logic of the target English question, guaranteeing semantic consistency. Simultaneously, the autonomous translation layer accurately transforms the target English question into a self-translated question. Finally, the target language reasoning layer generates a second target language answer that conforms to the usage habits of the target language, ensuring idiomatic expression. The complementary supervisory signals provided by the two types of answers generated through the dual paths further enhance... The learning and training process provides rich and effective optimization criteria, enabling the model to continuously reduce the deviation between the two-path answers during the iteration process. This avoids the semantic loss that easily occurs in the "direct translation followed by reasoning" of existing technologies, which leads to reasoning deviating from the core of the original problem. It also solves the problem of stiff expression in "single cross-language reasoning". At the same time, it significantly improves the model's adaptability and generalization ability to low-resource languages, even niche languages ​​and dialect expressions. There is no need to build a separate labeled dataset for each low-resource language, which greatly reduces the training cost and deployment threshold of multilingual models. Ultimately, the multilingual reasoning results generated by the target large language model take into account both logical accuracy and linguistic practicality, effectively breaking through the limitation of multilingual reasoning ability caused by the lack of training data for low-resource languages.

[0035] Based on any of the above embodiments, before processing the target English question through the cross-language reasoning layer to generate a first target language answer, the method further includes: Initial English questions are randomly selected from the English reasoning dataset; Based on the language prompts, the cross-language reasoning layer samples and answers the initial English question, and outputs multiple initial answers corresponding to the initial English question. The language prompts are used to guide the cross-language reasoning layer to answer the English question using the target language. Based on the standard answer, the accuracy rate of the answer to the initial English question is calculated according to the multiple initial answers, and the initial English questions with an accuracy rate greater than a predefined threshold are identified as target English questions.

[0036] Understandably, not all original English questions in English reasoning datasets are suitable for the model's current cross-language reasoning capabilities. Some questions are too difficult, have semantic complexity that exceeds the model's current understanding, or involve specialized expressions and logical structures that are less common in the target language. As a result, the model cannot reliably complete accurate reasoning and responses in the target language. If such questions are directly used in subsequent autonomous translation and target language reasoning training, it is very easy for the model to generate semantic biases, logically confused outputs, or even cause reasoning illusions, which seriously affects training stability and the final model performance.

[0037] It should be noted that statistically analyzing the accuracy of multiple initial answers more objectively reflects the model's stable cross-language reasoning ability for the question, eliminating pseudo-fit samples that are "correctly guessed once" and missed samples that are "incorrectly guessed once," and then combining this with a predefined threshold to complete precise screening, retaining only the initial English questions that the model can reliably answer correctly in the target language as the target English questions. This not only provides highly adapted core inputs for subsequent autonomous translation and target language reasoning, but also effectively avoids ineffective training and reasoning illusions, ensuring the training stability of the entire multilingual reasoning method and the synergistic improvement of the model's capabilities from the source.

[0038] The translation-based multilingual reasoning method provided in this invention, through multi-round parallel sampling of the cross-language reasoning layer, calculation of cross-language reasoning accuracy at the question level, and preset threshold filtering, retains only the questions that the model can answer correctly in the target language as the target English questions. This not only provides low-noise and highly adaptable training data for the subsequent autonomous translation layer and target language reasoning layer, but also constructs a dynamic course learning mechanism, allowing the model to start from samples within its own capabilities and gradually expand the training difficulty, thereby achieving a smooth iterative improvement in cross-language reasoning, autonomous translation, and target language reasoning capabilities.

[0039] Figure 3 This is the second flowchart of the translation-based multilingual reasoning method provided by the present invention, as shown below. Figure 3 As shown, step 105 also includes steps 1051-1053: Step 1051: Based on the standard answer, calculate the first stage reward value according to the first target language answer, and calculate the second stage reward value and the third stage reward value according to the second target language answer.

[0040] It should be noted that the first-stage reward value is calculated by comparing the first target language answer generated by the cross-language reasoning layer with the standard answer. The core measure is the semantic accuracy and logical consistency of the answer, that is, assessing whether the cross-language reasoning accurately reproduces the core intent and key information of the target English question, and the degree of matching between the answer and the standard answer in terms of reasoning logic and core conclusions. The reward value directly reflects the effectiveness of cross-language knowledge transfer. The second-stage reward value is calculated based on the second target language answer generated by the autonomous translation layer and the target language reasoning layer. It focuses on assessing the accuracy of the autonomous translation and the correctness of the question understanding, that is, verifying whether the self-translated question completely and accurately reproduces the semantics of the target English question, and thus judging whether the model's reasoning based on the self-translated question fits the core of the question. The reward value reflects the reliability of the autonomous translation stage. The third-stage reward value is also calculated based on the second target language answer, focusing on the naturalness, fluency, and practicality of the target language expression. That is, assessing whether the answer conforms to the grammatical rules and expression habits of the target language, and whether it avoids awkward translation. The reward value reflects the model's natural expression ability in the target language.

[0041] It should be noted that for the first and third stages, four rewards can be used in combination (such as format reward, language consistency reward, accuracy reward, and repetition reward), while the second stage is rewarded based on the accuracy of the answer in the second target language output in the third stage.

[0042] Understandably, by designing reward values ​​in stages and dimensions, reinforcement learning can optimize cross-language reasoning accuracy, autonomous translation reliability, and target language expression quality separately. The dual-path rewards complement each other, avoiding the bias of single-path optimization, and ultimately improving the overall performance of the model in multilingual reasoning.

[0043] Step 1052: Calculate the comprehensive reward value based on the first stage reward value, the second stage reward value, and the third stage reward value.

[0044] It should be noted that the comprehensive reward value can be calculated by linearly weighting the reward values ​​of the first stage, the second stage, and the third stage using preset weights. The specific calculation formula is: Comprehensive reward value = α × First stage reward value + β × Second stage reward value + γ × Third stage reward value. Wherein, α, β, and γ are preset weight coefficients, and satisfy α + β + γ = 1.

[0045] Step 1053: Based on the comprehensive reward value, train the large language model to obtain the target large language model.

[0046] It should be noted that, based on the feedback of three types of reward values, the cross-language reasoning layer, autonomous translation layer, and target language reasoning layer of the large language model are jointly updated by calculating the policy gradient. On the one hand, this strengthens the cross-language knowledge transfer capability of the cross-language reasoning layer and improves the semantic accuracy of the first target language answer. On the other hand, it optimizes the translation accuracy of the autonomous translation layer and the expressive fluency of the target language reasoning layer, reduces the deviation between the second target language answer and the standard answer, and makes the logic and expression of the dual-path answers more consistent.

[0047] The translation-based multilingual reasoning method provided in this invention effectively integrates the differentiated reward signals from the first, second, and third stages, avoiding the problem of insufficient reasoning accuracy or awkward expression caused by a single reward dimension dominating optimization. During iterative training, the model's cross-language reasoning layer, autonomous translation layer, and target language reasoning layer form a collaborative optimization closed loop, continuously enhancing the accuracy of cross-language knowledge transfer and constantly improving the reliability of autonomous translation and the naturalness of target language expression. This allows the answers generated by the dual paths to gradually converge in terms of logical consistency, semantic accuracy, and idiomatic expression. It can achieve a comprehensive improvement in multilingual reasoning ability without relying on large-scale low-resource language annotation data. The final target language model can stably output multilingual reasoning results with both logical rigor and linguistic practicality in low-resource language scenarios, effectively overcoming the bottleneck of limited multilingual reasoning ability caused by data deficiency in existing technologies.

[0048] Based on any of the above embodiments, the step of calculating a first-stage reward value based on the first target language answer, and calculating a second-stage reward value and a third-stage reward value based on the second target language answer, includes: Based on the standard answer, the first stage reward value is calculated according to the first format reward, the first language consistency reward, the first accuracy reward, and the first duplication reward of the first target language answer; Based on the standard answer, the second-stage reward value is calculated according to the accuracy rate of the answer in the second target language; Based on the standard answer, the second stage reward value is calculated according to the second format reward, second language consistency reward, second accuracy reward, and second repetition reward of the second target language answer, and then the third stage reward value is calculated.

[0049] It should be noted that the format reward (R) formatThe format reward is a reward dimension designed to standardize the model's output structure. Its core function is to force the model to follow a pre-defined specific output format. For example, it requires the model to wrap its reasoning process in labels, thereby ensuring that the model's reasoning logic is parsable, facilitating subsequent tracking, verification, and optimization of the thought process. In the reward calculation of the first and third stages, the format reward will verify the format compliance of the first target language answer and the second target language answer, respectively. If the output strictly conforms to the pre-defined format requirements, a positive reward will be given; if there are problems such as missing format or incorrect labels, the reward score will be reduced. This guides the model to maintain the consistency and standardization of the output format in cross-language reasoning and target language reasoning, providing a unified and parsable basic data for subsequent reward value calculation and model optimization.

[0050] It should be noted that the language consistency reward (R) lang The reward dimension is implemented based on a lightweight language detection tool. Its core function is to verify whether the model's output thought chain and final answer strictly use the target language throughout, avoiding language mixing issues during cross-language reasoning or target language reasoning, and ensuring the linguistic consistency and authenticity of the output content. In the reward calculation of the first and third stages, the language consistency reward evaluates the linguistic purity of the first target language answer and the second target language answer, respectively. If neither the thought chain nor the final answer shows non-target language mixing, a positive reward is given; if language mixing occurs, the reward score is reduced, thereby guiding the model to maintain the consistency of language expression in multilingual reasoning scenarios.

[0051] It should be noted that the accuracy bonus (R) acc The accuracy reward is a core reward dimension that uses the standard answer as a reference. Its core function is to directly measure the correctness of the model's reasoning results and is a key indicator for evaluating model performance. In the reward calculation of the first and third stages, the accuracy reward compares the matching degree between the first target language answer, the second target language answer, and the standard answer, respectively. If the output result is completely consistent with the standard answer, the highest reward score is given; if there is a deviation, the reward score is deducted according to the degree of deviation. In the reward calculation of the second stage, the reward directly uses the accuracy rate of the second target language answer as the core calculation basis, providing accurate feedback for the optimization of the autonomous translation and target language reasoning stages.

[0052] It should be noted that repeated rewards (R) repetitionThe reward dimension is based on sentence-level and n-gram-level detection. Its core function is to avoid the model generating redundant and repetitive content, ensuring the simplicity and information density of the output. In the reward calculation of the first and third stages, the repetition reward will detect the repetition of the first target language answer and the second target language answer, respectively. If sentence or phrase-level repetition is detected, the reward value is 0; if no repetition is detected, the reward value is 1, thereby guiding the model to generate more concise and efficient reasoning content.

[0053] The translation-based multilingual reasoning method provided in this invention employs a phased, multi-dimensional reward value calculation design. The first phase focuses on enhancing the standardization and accuracy of the cross-language reasoning layer output. The second phase concentrates on the core accuracy of autonomous translation and target language reasoning. The third phase comprehensively optimizes the multi-dimensional performance of the target language answer, providing layered and precise supervision signals for reinforcement learning. This allows the dual-path outputs to synergistically complement each other during iteration, effectively overcoming the limitations of low-resource language data deficiency, and ultimately improving the accuracy, standardization, and naturalness of the model's multilingual reasoning results.

[0054] Based on any of the above embodiments, training the large language model based on the comprehensive reward value to obtain the target large language model includes: Based on the comprehensive reward value, the large language model is trained using a group relative policy optimization algorithm to obtain the target large language model.

[0055] It should be noted that the group-relative policy optimization algorithm achieves efficient and stable training through the following mechanism: First, for each target English question, the current large language model needs to be driven to perform multiple policy samplings, generating multiple independent output groups. Each output group contains its generated first target language answer and second target language answer (i.e., multiple output groups for each target English question are generated through the cross-language inference layer, autonomous translation layer, and target language inference layer of the large language model). Subsequently, a comprehensive reward value is calculated for each output group according to a preset multi-dimensional reward function.

[0056] Understandably, the core of the group relative policy optimization algorithm lies in not directly using these original reward values ​​for optimization. Instead, it calculates the average and standard deviation of the rewards for all output groups, thus converting the reward of each output group into a "relative advantage score" relative to the average performance of that batch. Finally, based on this relative advantage score, the policy gradient is calculated, and the parameters of the large language model are updated to make it more inclined to produce outputs with higher relative advantages. By iteratively executing the above process, the model continuously optimizes through multiple rounds of self-play and comparison, eventually converging into the target large language model capable of generating high-quality, highly consistent multilingual inference results.

[0057] The translation-based multilingual reasoning method provided in this invention utilizes the Group Relative Policy Optimization (GRPO) algorithm to perform internal comparisons and optimizations using output groups generated from multiple samplings by the model itself. This method guides the co-evolution of the cross-language reasoning layer, the autonomous translation layer, and the target language reasoning layer with multi-stage, multi-dimensional reward signals (including format, linguistic purity, and answer accuracy). This effectively solves the problems of "language inconsistency" and lagging reasoning ability in low-resource languages, ultimately enabling the model to achieve robust and efficient multilingual deep reasoning capabilities relying solely on high-quality English data.

[0058] Based on any of the above embodiments, the step of training the large language model through reinforcement learning using a group relative policy optimization algorithm based on the comprehensive reward value to obtain the target large language model includes: The target English question is sampled multiple times using the large language model to generate multiple different output groups, where each output group includes a first target language answer and a second target language answer. Calculate the combined reward value for the first target language answer and the second target language answer in each output group; Based on the combined reward value of all output groups under the same target English question, calculate the relative advantage score of each output group; The strategy gradient is calculated based on the relative advantage score, and the model parameters of the large language model are iteratively updated based on the strategy gradient until the model converges, thus obtaining the target large language model.

[0059] It should be noted that the first target language answer in each output group is obtained through reasoning by the cross-language reasoning layer in the large language model, and the second target language answer in each output group is obtained through reasoning by the autonomous translation layer and the target language reasoning layer in the large language model.

[0060] In the specific implementation, for the same target English question, multiple independent samplings are performed by driving the large language model. Each sampling executes dual-path inference in parallel: one path directly generates the first target language answer through the cross-language inference layer, and the other path first transforms it into a self-translated question through the autonomous translation layer, and then passes it to the target language inference layer to produce the second target language answer, thus forming an output group containing two answers. Then, the comprehensive reward calculation and relative advantage evaluation stage begins: each output group calculates a comprehensive reward value based on a preset multi-dimensional reward function (covering format compliance, language purity, accuracy, and non-repetition). Next, the mean and standard deviation of the comprehensive reward values ​​of all output groups under the same target English question are calculated, and the comprehensive reward values ​​of each output group are converted into relative advantage scores based on the mean and standard deviation. Then, policy gradient updates are performed: based on the relative advantage scores of each output group, the parameter update direction is calculated through the policy gradient algorithm, which essentially guides the model to increase the probability of generating high-scoring output groups and decrease the probability of generating low-scoring output groups. Since the generation of each output group involves the collaboration of the cross-language inference layer, the autonomous translation layer, and the target language inference layer, this update simultaneously optimizes the collaboration mechanism between the three layers. Finally, through iterative and convergent cycles, the model continuously explores, evaluates, and optimizes itself in multiple rounds of self-play, gradually selecting and strengthening the optimal internal reasoning and expression strategies until its output stably reaches a high-performance state on multiple metrics, ultimately converging into a target large language model with powerful and robust multilingual deep reasoning capabilities.

[0061] In the specific implementation, the formula for calculating the relative advantage score of each output group is as follows: In the formula, R i This represents the total reward value of the i-th output group; σ represents the average of the combined reward values ​​of all output groups under the same target English problem; σ represents the sample standard deviation of the combined reward values ​​of all output groups under the same target English problem.

[0062] The translation-based multilingual reasoning method provided in this invention guides a large language model to sample the same English question multiple times, generating multiple output groups containing dual-path answers. It then calculates the relative advantage score between groups based on the comprehensive reward value, thereby driving strategy optimization. This mechanism, without relying on target language labeled data, forces the model to optimize its dual capabilities of direct cross-language reasoning and translation-before-reasoning through internal competition and self-comparison. This effectively solves the problems of "language inconsistency" and reasoning ability transfer in low-resource languages, ultimately training a target model capable of fluent, accurate, and target language-compliant complex reasoning.

[0063] The following describes the translation-based multilingual reasoning device provided by the present invention. The translation-based multilingual reasoning device described below corresponds to and can be referred to in conjunction with the translation-based multilingual reasoning method described above. For example... Figure 4 As shown, the translation-based multilingual reasoning device includes: The acquisition module 10 is used to acquire a large language model, wherein the large language model includes a cross-language reasoning layer, an autonomous translation layer, and a target language reasoning layer; The generation module 20 is used to process the target English question based on language prompts through the cross-language reasoning layer to generate a first target language answer; The generation module 20 is further configured to translate the target English question through the autonomous translation layer to generate a self-translated question; The generation module 20 is further configured to infer the self-translation problem through the target language inference layer and generate a second target language answer; Training module 30 is used to train the large language model based on the first target language answer and the second target language answer to obtain a target large language model, so as to generate multilingual reasoning results based on the target large language model.

[0064] Optionally, the generation module 20 is further configured to: Initial English questions are randomly selected from the English reasoning dataset; Based on the language prompts, the cross-language reasoning layer samples and answers the initial English question, and outputs multiple initial answers corresponding to the initial English question. The language prompts are used to guide the cross-language reasoning layer to answer the English question using the target language. Based on the standard answer, the accuracy rate of the answer to the initial English question is calculated according to the multiple initial answers, and the initial English questions with an accuracy rate greater than a predefined threshold are identified as target English questions.

[0065] Optionally, the training module 30 is further configured to: Based on the standard answer, the first stage reward value is calculated according to the first target language answer, and the second stage reward value and the third stage reward value are calculated according to the second target language answer; Calculate the comprehensive reward value based on the first stage reward value, the second stage reward value, and the third stage reward value; Based on the comprehensive reward value, the large language model is trained to obtain the target large language model.

[0066] Optionally, the training module 30 is further configured to: Based on the standard answer, the first stage reward value is calculated according to the first format reward, the first language consistency reward, the first accuracy reward, and the first duplication reward of the first target language answer; Based on the standard answer, the second-stage reward value is calculated according to the accuracy rate of the answer in the second target language; Based on the standard answer, the second stage reward value is calculated according to the second format reward, second language consistency reward, second accuracy reward, and second repetition reward of the second target language answer, and then the third stage reward value is calculated.

[0067] Optionally, the training module 30 is further configured to: Based on the comprehensive reward value, the large language model is trained using a group relative policy optimization algorithm to obtain the target large language model.

[0068] Optionally, the training module 30 is further configured to: The target English question is sampled multiple times using the large language model to generate multiple different output groups, where each output group includes a first target language answer and a second target language answer. Calculate the combined reward value for the first target language answer and the second target language answer in each output group; Based on the combined reward value of all output groups under the same target English question, calculate the relative advantage score of each output group; The strategy gradient is calculated based on the relative advantage score, and the model parameters of the large language model are iteratively updated based on the strategy gradient until the model converges, thus obtaining the target large language model.

[0069] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5As shown, the electronic device may include: a processor 510, a communications interface 520, a memory 530, and a communication bus 540, wherein the processor 510, communications interface 520, and memory 530 communicate with each other via the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute a translation-based multilingual reasoning method. This method includes: acquiring a large language model, wherein the large language model includes a cross-language reasoning layer, an autonomous translation layer, and a target language reasoning layer; processing a target English question based on language prompts through the cross-language reasoning layer to generate a first target language answer; translating the target English question through the autonomous translation layer to generate a self-translated question; reasoning about the self-translated question through the target language reasoning layer to generate a second target language answer; training the large language model based on the first and second target language answers to obtain a target large language model, thereby generating multilingual reasoning results based on the target large language model.

[0070] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0071] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the translation-based multilingual reasoning method provided by the above methods. The method includes: acquiring a large language model, wherein the large language model includes a cross-language reasoning layer, a self-translating layer, and a target language reasoning layer; processing a target English question through the cross-language reasoning layer based on language prompts to generate a first target language answer; translating the target English question through the self-translating layer to generate a self-translated question; reasoning about the self-translated question through the target language reasoning layer to generate a second target language answer; training the large language model based on the first target language answer and the second target language answer to obtain a target large language model, so as to generate a multilingual reasoning result based on the target large language model.

[0072] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the translation-based multilingual reasoning method provided by the above methods. The method includes: acquiring a large language model, wherein the large language model includes a cross-language reasoning layer, a self-translating layer, and a target language reasoning layer; processing a target English question through the cross-language reasoning layer based on language prompts to generate a first target language answer; translating the target English question through the self-translating layer to generate a self-translated question; reasoning about the self-translated question through the target language reasoning layer to generate a second target language answer; training the large language model based on the first target language answer and the second target language answer to obtain a target large language model, thereby generating a multilingual reasoning result based on the target large language model.

[0073] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0074] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0075] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A translation-based multilingual reasoning method, characterized in that, include: Obtain a large language model, wherein the large language model includes a cross-language reasoning layer, an autonomous translation layer, and a target language reasoning layer; Based on language prompts, the target English question is processed through the cross-language reasoning layer to generate a first target language answer; The target English question is translated using the autonomous translation layer to generate a self-translated question; The self-translation question is reasoned through the target language reasoning layer to generate a second target language answer; Based on the first target language answer and the second target language answer, the large language model is trained to obtain a target large language model, and multilingual reasoning results are generated based on the target large language model.

2. The translation-based multilingual reasoning method according to claim 1, characterized in that, Before processing the target English question through the cross-language reasoning layer to generate the first target language answer, the process further includes: Initial English questions are randomly selected from the English reasoning dataset; Based on the language prompts, the cross-language reasoning layer samples and answers the initial English question, and outputs multiple initial answers corresponding to the initial English question. The language prompts are used to guide the cross-language reasoning layer to answer the English question using the target language. Based on the standard answer, the accuracy rate of the answer to the initial English question is calculated according to the multiple initial answers, and the initial English questions with an accuracy rate greater than a predefined threshold are identified as target English questions.

3. The translation-based multilingual reasoning method according to claim 1, characterized in that, The step of training the large language model based on the first target language answer and the second target language answer to obtain the target large language model includes: Based on the standard answer, the first stage reward value is calculated according to the first target language answer, and the second stage reward value and the third stage reward value are calculated according to the second target language answer; Calculate the comprehensive reward value based on the first stage reward value, the second stage reward value, and the third stage reward value; Based on the comprehensive reward value, the large language model is trained to obtain the target large language model.

4. The translation-based multilingual reasoning method according to claim 3, characterized in that, The step of calculating a first-stage reward value based on the standard answer, according to the first target language answer, and calculating a second-stage reward value and a third-stage reward value based on the second target language answer, includes: Based on the standard answer, the first stage reward value is calculated according to the first format reward, the first language consistency reward, the first accuracy reward, and the first duplication reward of the first target language answer; Based on the standard answer, the second-stage reward value is calculated according to the accuracy rate of the answer in the second target language; Based on the standard answer, the second stage reward value is calculated according to the second format reward, second language consistency reward, second accuracy reward, and second repetition reward of the second target language answer, and then the third stage reward value is calculated.

5. The translation-based multilingual reasoning method according to claim 3, characterized in that, The process of training the large language model based on the comprehensive reward value to obtain the target large language model includes: Based on the comprehensive reward value, the large language model is trained using a group relative policy optimization algorithm to obtain the target large language model.

6. The translation-based multilingual reasoning method according to claim 5, characterized in that, The step of training the large language model through reinforcement learning using a group relative policy optimization algorithm based on the comprehensive reward value to obtain the target large language model includes: The target English question is sampled multiple times using the large language model to generate multiple different output groups, where each output group includes a first target language answer and a second target language answer. Calculate the combined reward value for the first target language answer and the second target language answer in each output group; Based on the combined reward value of all output groups under the same target English question, calculate the relative advantage score of each output group; The strategy gradient is calculated based on the relative advantage score, and the model parameters of the large language model are iteratively updated based on the strategy gradient until the model converges, thus obtaining the target large language model.

7. A translation-based multilingual reasoning device, characterized in that, include: The acquisition module is used to acquire a large language model, wherein the large language model includes a cross-language reasoning layer, an autonomous translation layer, and a target language reasoning layer; The generation module is used to process the target English question based on language prompts through the cross-language reasoning layer to generate a first target language answer; The generation module is also used to translate the target English question through the autonomous translation layer to generate a self-translated question; The generation module is also used to reason about the self-translation question through the target language reasoning layer to generate a second target language answer; The training module is used to train the large language model based on the first target language answer and the second target language answer to obtain a target large language model, so as to generate multilingual reasoning results based on the target large language model.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the translation-based multilingual reasoning method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the translation-based multilingual reasoning method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the translation-based multilingual reasoning method as described in any one of claims 1 to 6.

Images