Reordering method and apparatus based on noise channel model, and terminal device
By cascading forward translation models, language models, and backward translation models, a guiding model is generated and distilled, solving the problems of low translation efficiency and high resource consumption in existing technologies, and achieving efficient machine translation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SICHUAN LAN-BRIDGE INFORMATION TECHNOLOGY CO LTD
- Filing Date
- 2023-01-12
- Publication Date
- 2026-05-12
AI Technical Summary
Existing reordering methods based on noisy channel models are inefficient and resource-intensive when using the target language's language model and reverse translation model.
By concatenating a forward translation model, a language model, and a reverse translation model, a guiding model is generated. A first learning model and a second learning model are obtained through distillation. Pseudo-labeled training samples are used to correct and optimize parameters, generating translations with fluency and accuracy.
While ensuring translation quality, it improves the efficiency of machine translation and reduces resource consumption.
Smart Images
Figure CN116306693B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of machine translation technology, and in particular to a reordering method, apparatus, and terminal device based on a noisy channel model. Background Technology
[0002] In noisy channel models of machine translation, three models are typically used to construct the re-ranking score: a forward translation model, a backward translation model, and a target language language model. The target language language model measures the fluency of the translation, while the backward translation model measures its accuracy. However, if the translation contains mistranslations, omissions, or overtranslations, it is usually difficult to translate it back to the original text using the backward translation model. Therefore, when translating using the target language language model and the backward translation model, the translation is further re-ranked, a process known as re-ranking.
[0003] Currently, the commonly used reordering method is based on a noisy channel model. However, this noisy channel model generates several unordered translations for each final output translation, and each unordered translation requires a fluency and accuracy score calculation. This makes the translation time tens or even hundreds of times longer than that of a regular translation model, ultimately reducing the efficiency of online machine translation services. Furthermore, this changes the deployment process from a single translation model to simultaneously deploying three models: a regular translation model, a language model, and a reverse translation model, resulting in extremely high resource consumption. Summary of the Invention
[0004] The main objective of this invention is to propose a reordering method, apparatus, and terminal device based on a noisy channel model, which solves the problems of low translation efficiency and high resource consumption in current reordering methods when using the target language's language model and reverse translation model.
[0005] To achieve the above objectives, a first aspect of the present invention provides a reordering method based on a noisy channel model, wherein the noisy channel model includes a cascaded forward translation model, a language model, and a reverse translation model, and the method includes:
[0006] Obtain the original text to be translated, the original translation, the output of the forward translation model, the output of the language model, and the output of the backward translation model;
[0007] A parameter search is performed on the noisy channel model to generate a guided model; the parameter search includes searching for the weights of the language model, the weights of the back translation model, and the weights of the length penalty term, based on maximizing the bleu value of the noisy channel model.
[0008] The guided model is used to predict the original text to be translated and generate pseudo-labeled translations.
[0009] The guided model is distilled using the original training samples and pseudo-labeled training samples to obtain a first learning model. The original training samples include the original text to be translated and the original translation. The pseudo-labeled training samples include the original text to be translated and the pseudo-labeled translation.
[0010] The ordinary model is processed based on the parameters of the first learning model to obtain a second learning model for re-ranking.
[0011] In conjunction with the first aspect of the present invention, in a first embodiment, a first learning model is obtained by distilling the guided model using original training samples and pseudo-labeled training samples, including:
[0012] When distilling the pilot model, a weight mapper is constructed in the original learning model to map the weights in the pilot model to the original learning model to obtain the first learning model.
[0013] In conjunction with the first embodiment of the first aspect of the present invention, in the second embodiment of the present invention, a common model is processed according to the parameters of the first learning model to obtain a second learning model for re-ranking, including:
[0014] By mapping parameters, the parameters in the first learning model are transferred to the ordinary model to obtain the second learning model.
[0015] In conjunction with the second embodiment of the first aspect of the present invention, in the third embodiment of the present invention, after processing the ordinary model according to the parameters of the first learning model to obtain the second learning model for re-ranking, the method includes:
[0016] Using the original training data and pseudo-labeled data, the second learning model is fine-tuned again using ordinary distillation.
[0017] In conjunction with the first aspect of the present invention, in the fourth embodiment of the present invention, parameter search is performed on the noise channel model to generate a guiding model, including:
[0018] The forward translation model in the noisy channel model is corrected by using the searched parameters to obtain the guiding model.
[0019] In conjunction with the fourth embodiment of the first aspect of the present invention, in the fifth embodiment of the present invention, the parameter correction method is weight alignment.
[0020] In conjunction with the first aspect of the present invention, in the sixth embodiment of the present invention, based on maximizing the BLUE value of the noise channel model, the weights of the language model, the weights of the back translation model, and the weights of the length penalty term are searched, as follows:
[0021] ;
[0022] Among them, Ls L represents the length of the original text to be translated. t λ1 is the length of the original translation, λ2 is the weight of the language model, λ3 is the weight of the reverse translation model, lenpen is the weight of the length penalty term, t is the original text to be translated, and s is the original translation.
[0023] A second aspect of this invention provides a reordering apparatus based on a noisy channel model, wherein the noisy channel model includes a cascaded forward translation model, a language model, and a reverse translation model, and the apparatus includes:
[0024] The training data acquisition module is used to acquire the original text to be translated, the original translation text, the output of the forward translation model, the output of the language model, and the output of the backward translation model.
[0025] The parameter search module is used to perform parameter search on the noisy channel model to generate a guided model. The parameter search includes searching for the weights of the language model, the weights of the back translation model, and the weights of the length penalty term, based on maximizing the bleu value of the noisy channel model.
[0026] The pseudo-label translation acquisition module is used to predict the original text to be translated using the guidance model and generate pseudo-label translations.
[0027] The model distillation module is used to distill the guided model using the original training samples and pseudo-labeled training samples to obtain a first learning model. The original training samples include the original text to be translated and the original translation, and the pseudo-labeled training samples include the original text to be translated and the pseudo-labeled translation.
[0028] The second learning model acquisition module is used to process the ordinary model according to the parameters of the first learning model to obtain a second learning model for re-ranking.
[0029] A third aspect of the present invention provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method provided in the first aspect above.
[0030] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method provided in the first aspect above.
[0031] This invention provides a reordering method based on a noisy channel model. It corrects the original noisy channel model through parameter search to generate a guiding model. Then, through distillation, the original learning model corresponding to the guiding model learns the output of the guiding model, resulting in an improved first learning model and a second learning model. The second learning model generates a translation similar to the pseudo-labeled translation output by the guiding model. These are then reordered, and the reordered output, which is the final translation, also possesses the fluency and accuracy of the guiding model. The model distillation method eliminates the need for multiple calculations of fluency and accuracy scores, enabling the second learning model to achieve the fluency and accuracy of the guiding model. Therefore, this invention improves the efficiency of machine translation while maintaining translation quality. Attached Figure Description
[0032] Figure 1 This is a schematic diagram illustrating the implementation process of the reordering method based on a noisy channel model provided in an embodiment of the present invention.
[0033] Figure 2 This is a schematic diagram of the series structure of the noise channel model provided in an embodiment of the present invention;
[0034] Figure 3 This is a schematic diagram of the composition structure of a reordering device based on a noise channel model provided in an embodiment of the present invention.
[0035] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0036] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
[0037] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.
[0038] In this document, suffixes such as "module," "part," or "unit" used to denote elements are used only for the purpose of illustrative purposes and do not have any specific meaning in themselves. Therefore, "module" and "part" can be used interchangeably.
[0039] like Figure 1As shown, embodiments of the present invention provide a reordering method based on a noisy channel model, and as... Figure 2 As shown, in this embodiment of the invention, the noise channel model 20 includes a forward translation model 21, a language model 22, and a reverse translation model 23 connected in series. In one embodiment, the series connection is such that the forward translation model 21 is connected to the language model 22 and the reverse translation model 23 respectively.
[0040] In practical applications, the forward translation model is exemplarily the zh-en fw model; the language model is a general language model used to translate the original text to be translated, exemplarily the en LM model; and the reverse translation model is exemplarily the zh-en bt model.
[0041] By cascading forward translation, language, and backward translation models, the noisy channel model is biased towards generating fluent and backward-translatable original translations. Based on this model reordering, the fluency and accuracy of the final translation are guaranteed.
[0042] Figure 1 In this context, the reordering method based on the noisy channel model includes, but is not limited to, the following steps:
[0043] S101. Obtain the original text to be translated, the original translation, the output of the forward translation model, the output of the language model, and the output of the reverse translation model.
[0044] In step S101 above, the original translation is the translation corresponding to the original text to be translated, which is pre-set training data.
[0045] S102. Perform parameter search on the noise channel model to generate a guiding model.
[0046] The parameter search includes searching for the weights of the language model, the weights of the reverse translation model, and the weights of the length penalty term, based on maximizing the bleu value of the noisy channel model.
[0047] In this embodiment of the invention, parameter search is performed on the noisy channel model to generate a guiding model. In practice, the forward translation model in the noisy channel model is modified by using the searched parameters, namely the weights of the language model, the weights of the reverse translation model, and the weights of the length penalty term searched in the above steps, to obtain the guiding model.
[0048] The detailed implementation of parameter correction is weight alignment.
[0049] For example, in step S102 above, based on maximizing the bleu value of the noise channel model, the weights of the search language model, the weights of the reverse translation model, and the weights of the length penalty term are calculated using the following formula:
[0050] ;
[0051] Among them, L s L represents the length of the original text to be translated. t λ1 is the length of the original translation, λ2 is the weight of the language model, λ3 is the weight of the reverse translation model, lenpen is the weight of the length penalty term, t is the original text to be translated, and s is the original translation.
[0052] S103. The original text to be translated is predicted using the guidance model to generate pseudo-labeled translations;
[0053] S104. Using the original training samples and pseudo-labeled training samples, distill the bootstrap model to obtain the first learning model.
[0054] In step S104 above, the original training samples include the original text to be translated and the original translation, and the pseudo-label training samples include the original text to be translated and the pseudo-label translation.
[0055] S105. Process the ordinary model according to the parameters of the first learning model to obtain a second learning model for re-ranking.
[0056] In this embodiment of the invention, the model distillation methods in steps S104 and S105 include a three-step distillation method of weight distillation-parameter transfer-ordinary distillation, and a method of directly using ordinary distillation. Both methods can obtain the first learning model.
[0057] This invention provides a detailed description of a three-step distillation method using weighted distillation-parameter transfer-ordinary distillation. Specifically, when using this three-step method, the detailed implementation of step S104 is as follows:
[0058] When distilling the pilot model, a weight mapper is constructed in the original learning model to map the weights in the pilot model to the original learning model to obtain the first learning model.
[0059] The detailed implementation method of the above step S105 is as follows:
[0060] By mapping parameters, the parameters in the first learning model are transferred to the ordinary model to obtain the second learning model.
[0061] In one embodiment, after processing a regular model based on the parameters of the first learning model to obtain a second learning model for re-ranking, the method further includes:
[0062] Using the original training data and pseudo-labeled data, the second learning model is fine-tuned again using ordinary distillation.
[0063] This invention corrects the original noisy channel model through parameter search to generate a guiding model. Then, through distillation, the original learning model corresponding to the guiding model learns the output of the guiding model, resulting in an improved first and second learning model. The second learning model generates a translation similar to the pseudo-labeled translation output by the guiding model. These are then reordered, and the reordered output, which is the final translation, also possesses the fluency and accuracy of the guiding model. The model distillation method eliminates the need for multiple calculations of fluency and accuracy scores, enabling the second learning model to achieve the same fluency and accuracy as the guiding model. Therefore, this invention improves the efficiency of machine translation while maintaining translation quality.
[0064] like Figure 3 As shown, this embodiment of the invention also provides a reordering device 30 based on a noise channel model, the noise channel model being similarly described above. Figure 2 As shown, it includes a cascaded forward translation model 21, a language model 22, and a reverse translation model 23.
[0065] Figure 3 In this context, the reordering device based on the noisy channel model includes:
[0066] The training data acquisition module 31 is used to acquire the original text to be translated, the original translation text, the output of the forward translation model, the output of the language model, and the output of the reverse translation model.
[0067] The parameter search module 32 is used to perform parameter search on the noisy channel model to generate a guided model; the parameter search includes searching for the weights of the language model, the weights of the back translation model, and the weights of the length penalty term based on maximizing the bleu value of the noisy channel model.
[0068] The pseudo-label translation acquisition module 33 is used to predict the original text to be translated through the guidance model and generate pseudo-label translations.
[0069] The model distillation module 34 is used to distill the guided model using the original training samples and pseudo-labeled training samples to obtain a first learning model. The original training samples include the original text to be translated and the original translation, and the pseudo-labeled training samples include the original text to be translated and the pseudo-labeled translation.
[0070] The second learning model acquisition module 35 is used to process the ordinary model according to the parameters of the first learning model to obtain a second learning model for re-ranking.
[0071] This invention also provides a terminal device including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements each step of the reordering method based on the noise channel model as described in the above embodiments.
[0072] This invention also provides a storage medium, which is a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements the various steps in the reordering method based on the noise channel model as described in the above embodiments.
[0073] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the foregoing embodiments have described the present invention in detail, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.
Claims
1. A reordering method based on a noisy channel model, characterized in that, The noise channel model includes a cascaded forward translation model, a language model, and a reverse translation model, and the method includes: Obtain the original text to be translated, the original translation, the output of the forward translation model, the output of the language model, and the output of the backward translation model; A parameter search is performed on the noisy channel model to generate a guided model; the parameter search includes searching for the weights of the language model, the weights of the back translation model, and the weights of the length penalty term, based on maximizing the bleu value of the noisy channel model. The guided model is used to predict the original text to be translated and generate pseudo-labeled translations. The guided model is distilled using the original training samples and pseudo-labeled training samples to obtain a first learning model. The original training samples include the original text to be translated and the original translation. The pseudo-labeled training samples include the original text to be translated and the pseudo-labeled translation. The ordinary model is processed based on the parameters of the first learning model to obtain a second learning model for re-ranking. Among them, based on maximizing the BLUE value of the noise channel model, the weights of the search language model, the weights of the back translation model, and the weights of the length penalty term are calculated using the following formula: ； Among them, L s L represents the length of the original text to be translated. t λ1 is the length of the original translation, λ2 is the weight of the language model, λ3 is the weight of the reverse translation model, lenpen is the weight of the length penalty term, t is the original text to be translated, and s is the original translation.
2. The reordering method based on a noisy channel model as described in claim 1, characterized in that, Using the original training samples and pseudo-labeled training samples, the bootstrap model is distilled to obtain the first learning model, including: When distilling the pilot model, a weight mapper is constructed in the original learning model to map the weights in the pilot model to the original learning model to obtain the first learning model.
3. The reordering method based on a noisy channel model as described in claim 2, characterized in that, The ordinary model is processed based on the parameters of the first learning model to obtain a second learning model for re-ranking, including: By mapping parameters, the parameters in the first learning model are transferred to the ordinary model to obtain the second learning model.
4. The reordering method based on a noisy channel model as described in claim 3, characterized in that, After processing the ordinary model based on the parameters of the first learning model to obtain the second learning model for re-ranking, the process includes: Using the original training data and pseudo-labeled data, the second learning model is fine-tuned again using ordinary distillation.
5. The reordering method based on a noisy channel model as described in claim 1, characterized in that, Perform parameter search on the noisy channel model to generate a guided model, including: The forward translation model in the noisy channel model is corrected by using the searched parameters to obtain the guiding model.
6. The reordering method based on a noisy channel model as described in claim 5, characterized in that, The parameter correction method is weight alignment.
7. A reordering device based on a noisy channel model, characterized in that, The noise channel model includes a cascaded forward translation model, a language model, and a reverse translation model, and the device includes: The training data acquisition module is used to acquire the original text to be translated, the original translation text, the output of the forward translation model, the output of the language model, and the output of the backward translation model. The parameter search module is used to perform parameter search on the noisy channel model to generate a guided model. The parameter search includes searching for the weights of the language model, the weights of the back translation model, and the weights of the length penalty term, based on maximizing the bleu value of the noisy channel model. The pseudo-label translation acquisition module is used to predict the original text to be translated using the guidance model and generate pseudo-label translations. The model distillation module is used to distill the guided model using the original training samples and pseudo-labeled training samples to obtain a first learning model. The original training samples include the original text to be translated and the original translation, and the pseudo-labeled training samples include the original text to be translated and the pseudo-labeled translation. The second learning model acquisition module is used to process the ordinary model according to the parameters of the first learning model to obtain the second learning model for re-ranking. Among them, based on maximizing the BLUE value of the noise channel model, the weights of the search language model, the weights of the back translation model, and the weights of the length penalty term are calculated using the following formula: ； Among them, L s L represents the length of the original text to be translated. t λ1 is the length of the original translation, λ2 is the weight of the language model, λ3 is the weight of the reverse translation model, lenpen is the weight of the length penalty term, t is the original text to be translated, and s is the original translation.
8. A terminal device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements each step of the reordering method based on a noisy channel model as described in any one of claims 1 to 6.
9. A storage medium, said storage medium being a computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements each step of the reordering method based on a noisy channel model as described in any one of claims 1 to 6.