Training method and device for translator behavior learning-oriented multi-language machine translation large model and medium
The training method enhances multilingual translation and APE capabilities in large language models by using a carefully constructed dataset and React dataset for translator behavior learning, addressing the limitations of existing models.
Patent Information
- Application Number
- CN202411283492.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-13
- Publication Date
- 2024-12-31
AI Technical Summary
Existing large language models suffer from poor multilingual translation capabilities, poor translation quality, and lack of Auto Post Editing (APE) capabilities, particularly when interacting with multiple prompts to achieve high-quality translations.
A training method for a large multilingual machine translation model involves constructing a multilingual dataset with specific data proportions, using a transformer model for multi-stage training, and incorporating a React dataset for translator behavior learning with Lora fine-tuning to enhance self-reflection and tool-using capabilities.
The method improves multilingual translation capabilities and translation quality, enabling effective APE by simulating translator behavior, resulting in better translation outcomes compared to existing models.
Smart Images
Figure CN119227765A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of machine translation, and in particular, relates to a training method, device and medium for a large multilingual machine translation model oriented to translator behavior learning. Background Art
[0002] With the emergence of large language models such as chatGPT, it has become possible to solve all tasks in natural language processing, such as text understanding and text generation, through the same model. In these large language models such as chatGPT, Llama, Bloom, etc., in addition to being able to complete conventional conversations and text generation tasks, researchers have also begun to pay attention to their translation capabilities. Under this "grand unified" framework, machine translation tasks are also about to undergo a transformation, and using large language models to achieve high-quality machine translation has become a hot research direction.
[0003] However, existing large language models have three problems: (1) Poor multilingual translation capabilities: For example, chatGPT and llama are usually trained with English corpus as the core, which has language bias problems. Corpus in other languages is usually relatively small, which also leads to poor multilingual translation capabilities; (2) Poor translation quality: Compared with specialized neural machine translation models, some known large language models usually have difficulty exceeding neural machine translation models in terms of evaluation results (such as Bleu values) on standard machine translation test sets. In addition, according to manual evaluation results, large language models also have more serious omissions when performing machine translation than neural machine translation models. (3) Lack of APE (Auto Post Editing) capability: When interacting with a large language model through prompts and other means to complete machine translation, if you want to obtain high-quality translations, you need to interact with the large language model multiple times and let the large language model make multiple modifications to obtain relatively high-quality translations. Summary of the invention
[0004] The purpose of the present invention is to provide a training method for a large multilingual machine translation model oriented to translator behavior learning, so as to solve the technical problems existing in the prior art.
[0005] In order to achieve the above object, the technical solution adopted by the present invention is as follows: A training method for a large multilingual machine translation model for translator behavior learning includes the following steps: Step S1: Multilingual training dataset: Build a large language model training dataset that integrates multiple data. The dataset contains three types of data: (11) English monolingual data, accounting for 50%; (12) Multilingual monolingual data: Monolingual data of multiple languages, accounting for 40%; (13) Multilingual alignment data: By constructing a specific prompt, bilingual and multilingual machine translation data are converted into large model training data, accounting for 10%; Step S2: Using the transformer model as the model base, perform multi-stage multi-language large model training and output the multi-language large model after preliminary training: (21) Sort the training data by difficulty. The sorting order is from simple to difficult: general knowledge, multilingual knowledge, and cross-language knowledge. (22) Model training process: In the first stage, the general knowledge learning stage, 100% of the English monolingual data is used to train the multilingual large model; in the second stage, the multilingual knowledge learning stage, multilingual monolingual data is gradually added, so that the proportion of English monolingual data is gradually reduced from 100% to 55%, and the proportion of multilingual monolingual data is gradually increased from 0% to 45%; in the third stage, the cross-lingual knowledge learning stage, multilingual aligned data is gradually added, so that the proportion of English monolingual data is gradually reduced from 55% to 50%, the proportion of multilingual monolingual data is gradually reduced from 45% to 40%, and the proportion of multilingual aligned data is gradually increased from 0% to 10%; (23) Complete the training of the large language model and output the multi-language large model after preliminary training; Step S3: Model training for translator behavior learning: (31) Build a React dataset; (32) Based on the React dataset, on the basis of the multilingual large model after preliminary training, Lora fine-tuning is used to update some parameters of the multilingual large model. The updating method is as follows: add a bypass branch to the Q, K, and V matrices in the multi-head self-attention mechanism. During the training process using the React data, freeze the parameters of the original multilingual large model and update the parameters of the bypass branch. After the training is completed, load the parameters of the bypass branch into the original multilingual large model. (33) Output a large multilingual translator model.
[0006] Preferably, it is characterized in that the (12) multilingual and monolingual data includes: based on English, French, Russian, Spanish, German, Japanese, Korean, Thai, Portuguese, Cambodian, Indonesian, Italian, Arabic, and Vietnamese, open source corpora and machine-translated monolingual corpora are collated to form multilingual and monolingual data.
[0007] Preferably, the method of gradually adding the multilingual monolingual data / gradually adding the multilingual aligned data is to add them randomly in proportion.
[0008] Preferably, the original multilingual large model refers to a multilingual large model after preliminary training.
[0009] Preferably, the React dataset is generated by chatGPT.
[0010] In order to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium on which a computer program is stored, characterized in that the computer program is executed by a processor to implement the training method of the multilingual machine translation large model for translator behavior learning.
[0011] In order to achieve the above-mentioned purpose, the present invention also provides a device for training a large multilingual machine translation model for translator behavior learning, comprising: a processor and a memory; the memory is used to store a computer program; the processor is connected to the memory, and is used to execute the computer program stored in the memory, so that the device for training a large multilingual machine translation model for translator behavior learning executes the training method for a large multilingual machine translation model for translator behavior learning.
[0012] Compared with the prior art, the present invention has the following beneficial effects: The present invention constructs a large model based on multilingual machine translation. The model is based on the transformer model and is trained on a carefully constructed multilingual dataset. Then, a React dataset for simulating translator behavior is generated through chatGPT. The model is trained based on the React data to enhance the model's self-reflection and tool-using capabilities. Compared with the prior art, the present invention has the advantages of better multilingual translation capabilities, good translation quality, and APE capabilities. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] Figure 1 This is a schematic diagram of the principle of Example 1 of the present invention.
[0014] Figure 2 This is a principle block diagram of Example 3 of the present invention.
[0015] Figure 3 This is the translation process of ChatGPT in the prior art.
[0016] Figure 4 This is the second translation process of ChatGPT in the prior art.
[0017] Figure 5 This is the translation process three of ChatGPT in the prior art.
[0018] Figure 6 The present invention provides a translation process using a large multi-language machine translation model for translator behavior learning. DETAILED DESCRIPTION
[0019] In order to enable those skilled in the art to have a clearer understanding and knowledge of the present invention, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described below are only used to explain the present invention, and are convenient for understanding. The technical solutions provided by the present invention are not limited to the technical solutions provided by the following embodiments, and the technical solutions provided by the embodiments should not limit the protection scope of the present invention.
[0020] It should be noted that the illustrations provided in the following embodiments are only used to illustrate the basic concept of the present invention in a schematic manner. Therefore, the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the form, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.
[0021] Example 1 like Figure 1 As shown, this embodiment provides a training method for a large multilingual machine translation model for translator behavior learning. The model training is based on a transformer model, preferably a decoder-only transformer.
[0022] In this embodiment, the following steps are specifically included: 1. Step S1: Multilingual training dataset Construct a large language model training dataset that integrates multiple data. The dataset contains three types of data: (11) English monolingual data, accounting for 50%; (12) Multilingual monolingual data: monolingual data of multiple languages, accounting for 40%; (13) Multilingual alignment data: by constructing a specific prompt, bilingual and multilingual machine-translated data are converted into large model training data, accounting for 10%; Among them, multilingual monolingual data includes: based on English, French, Russian, Spanish, German, Japanese, Korean, Thai, Portuguese, Cambodian, Indonesian, Italian, Arabic, and Vietnamese, open source corpora and machine-translated monolingual corpora are sorted to form multilingual monolingual data. It should be noted that multilingual data is not limited to the above languages, and other low-resource languages can also be used or expanded.
[0023] Step S2: Multi-stage large language model training: In order to make it easier for the model to learn relevant knowledge, the data difficulty is sorted according to a heuristic method, namely: general knowledge (simple), multilingual knowledge (medium), cross-language knowledge (most difficult), and the training data is sorted according to the difficulty of the data. During the training process, the data is gradually sent to the model for training from simple to most difficult. The specific method is as follows: In the first stage, the basic model is used as the initial model. In the general knowledge learning stage, 100% of the English monolingual data is used to train the multilingual large model. For the convenience of distinction, the model after the training in this stage can be named the English general large model; in the second stage In the multilingual knowledge learning stage, multilingual and monolingual data are gradually added, so that the proportion of English monolingual data is gradually reduced from 100% to 55%, and the proportion of multilingual and monolingual data is gradually increased from 0% to 45%. For the convenience of distinction, the model trained in this stage can be named the multilingual general large model; in the third stage, the cross-language knowledge learning stage, multilingual aligned data are gradually added, so that the proportion of English monolingual data is gradually reduced from 55% to 50%, the proportion of multilingual and monolingual data is gradually reduced from 45% to 40%, and the proportion of multilingual aligned data is gradually increased from 0% to 10%. For the convenience of distinction, the model trained in this stage can be named the multilingual translation large model.
[0024] In this step, the method of gradually adding multilingual monolingual data / gradually adding multilingual aligned data is to add them randomly in proportion. For example, taking the gradual addition of multilingual monolingual data as an example, the ultimate goal is to gradually reduce the proportion of English monolingual data from 100% to 55%, and gradually increase the proportion of multilingual monolingual data from 0% to 45%. The gradual addition process is: first, add multilingual monolingual data, accounting for 5%, and English monolingual data accounting for 95%; second, add multilingual monolingual data, accounting for 10%, and English monolingual data accounting for 90%, and so on.
[0025] The training of the large language model is completed, and the multilingual large model after preliminary training is output, which is the multilingual translation large model mentioned above, and is used as the original multilingual large model for training in step S3.
[0026] Step S3: Model training for translator behavior learning: In order to enable the multilingual translation model to have the same post-editing capabilities as human translators, a React dataset for simulating translator behavior was generated through chatGPT to train the multilingual translation model to simulate the ability to use tools in the post-editing process, so as to improve the model's self-reflection and tool-using capabilities. The specific prompts are as follows: System: Assuming you are a professional translator, you now have access to the following tools: qe: qe(src_sentence, tgt_sentence) -> list of str - This is a tool that can evaluate the quality of machine translation text. It inputs the original text and the machine translation text, and outputs the possible errors in the translation. args: {"src_sentence": {"title": "src_sentence", "type": "str"}, "tgt_sentence": {"title": "tgt_sentence", "type": "str"}} pe: pe(tgt_sentence, qe_result) -> str - This is a tool that can modify machine translation text. Input machine translation text and possible error positions, modify the machine translation text, and output the modified translation. args: {"tgt_sentence": {"title": "tgt_sentence", "type": "str"}, qe_result:{"title": "qe_result", "type": "list(str)"}} polish: polish(tgt_sentence) -> str - This is a tool that can polish machine translated text. Input machine translated text and output polished machine translated text. args: {"tgt_sentence": {"title": "tgt_sentence", "type": "str"}} To specify a tool, use a json form by providing an action (tool name) and an action_input (tool input).
[0027] Valid "action" values are "final answer" or "qe", "pe", "polish" As shown in the figure, each json can only provide one operation: ``` { "action": $TOOL_NAME "action_input": $INPUT } ``` Follow the format below Question: Enter the question you want to answer Think: Consider the steps before and after action: ```.
[0028] Based on the React dataset, on the basis of the multilingual large model after preliminary training, Lora fine-tuning is used to update some parameters of the multilingual large model. The updating method is as follows: add bypass branches to the Q, K, and V matrices in the multi-head self-attention mechanism. During the training process using React data, freeze the parameters of the original multilingual large model and update the parameters of the bypass branch. After training, load the parameters of the bypass branch into the original multilingual large model; finally, output the multilingual translator large model, which is a multilingual machine translation large model for translator behavior learning.
[0029] Compared with the prior art, the technical solution provided by this embodiment has the advantages of better multilingual translation capability, good translation quality, and APE capability.
[0030] A comparison example between the technical solution provided in this embodiment and ChatGPT translation is shown in the figure below. Figures 3 to 6 As shown, Figures 3-5 The process of translating for ChatGPT, Figure 6 This is the translation process of the technical solution provided in this embodiment; as can be seen from the figure, the ChatGPT translation process has the following problems: QE: quality assessment failure, polishing: omissions; and the technical solution provided in this embodiment effectively solves the above problems.
[0031] Example 2 This embodiment provides a computer-readable storage medium on which a computer program is stored, and the computer program is executed by a processor to implement the training method of a large multilingual machine translation model for translator behavior learning provided in Example 1. A person of ordinary skill in the art can understand that all or part of the steps of implementing the method provided in Example 1 can be completed by hardware related to the computer program, and the above-mentioned computer program can be stored in a computer-readable storage medium. When the program is executed, the steps of the method provided in Example 1 are executed; and the above-mentioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes.
[0032] Example 3 like Figure 2 As shown, this embodiment provides a training device for a large multilingual machine translation model for translator behavior learning, comprising: a processor and a memory; the memory is used to store a computer program; the processor is connected to the memory, and is used to execute the computer program stored in the memory, so that the training device for a large multilingual machine translation model for translator behavior learning executes the training method for a large multilingual machine translation model for translator behavior learning provided in Example 1.
[0033] Specifically, the memory includes: ROM, RAM, disk, USB flash drive, memory card or CD and other media that can store program codes.
[0034] Preferably, the processor can be a general-purpose processor, including a central processing unit, a network processor, etc.; it can also be a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component.
[0035] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.
Claims
1. A training method for a large multilingual machine translation model for translator behavior learning, characterized in that: The following steps are involved: Step S1: Multilingual training data set: Build a large language model training data set that integrates multiple data. The data set includes three aspects of data: (11) English monolingual data, accounting for 50%; (12) Multilingual monolingual data: Monolingual data of multiple languages, accounting for 40%; (13) Multilingual alignment data: By constructing a specific translation prompt, the specific construction method is to manually generate some basic prompt examples, and then use a commercial large language model to generate several different prompts. Convert the existing bilingual and multilingual machine translation data into large model training data, accounting for 10% of the data; Step S2: Using the transformer model as the model base, perform multi-stage multi-language large model training and output the multi-language large model after preliminary training: (21) Sort the training data by difficulty. The sorting order is from simple to difficult: general knowledge, multilingual knowledge, and cross-language knowledge. (22) Model training process: In the first stage, the general knowledge learning stage, 100% of the English monolingual data is used to train the multilingual large model; In the second stage, the multilingual knowledge learning stage, multilingual and monolingual data were gradually added, so that the proportion of English monolingual data gradually decreased from 100% to 55%, and the proportion of multilingual and monolingual data gradually increased from 0% to 45%; In the third stage, cross-language knowledge learning, multilingual aligned data was gradually added, so that the English monolingual data was gradually reduced from 55% to 50%, the multilingual monolingual data was gradually reduced from 45% to 40%, and the multilingual aligned data was gradually increased from 0% to 10%; (23) Complete the training of the large language model and output the multi-language large model after preliminary training; Step S3: Model training for translator behavior learning: (31) Build a React dataset; (32) Based on the React dataset, on the basis of the multilingual large model after preliminary training, Lora fine-tuning is used to update some parameters of the multilingual large model. The updating method is as follows: add a bypass branch to the Q, K, and V matrices in the multi-head self-attention mechanism. During the training process using the React data, freeze the parameters of the original multilingual large model and update the parameters of the bypass branch. After the training is completed, load the parameters of the bypass branch into the original multilingual large model. (33) Output a large multilingual translator model.
2. The training method for a large multilingual machine translation model for translator behavior learning according to claim 1 is characterized in that: The (12) multilingual and monolingual data include: based on English, French, Russian, Spanish, German, Japanese, Korean, Thai, Portuguese, Cambodian, Indonesian, Italian, Arabic, and Vietnamese, the open source corpus and the machine-translated monolingual corpus are collated to form the multilingual and monolingual data.
3. The training method for a large multilingual machine translation model for translator behavior learning according to claim 2 is characterized in that: The method of gradually adding multilingual monolingual data / gradually adding multilingual aligned data is to add them randomly in proportion.
4. The training method for a large multilingual machine translation model for translator behavior learning according to claim 3 is characterized in that: The original multilingual large model refers to the multilingual large model after preliminary training.
5. The training method for a large multilingual machine translation model for translator behavior learning according to claim 4 is characterized in that: The React dataset is generated by chatGPT.
6. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer program is executed by a processor to implement the training method for a large multilingual machine translation model for translator behavior learning as described in any one of claims 1 to 5.
7. A device for training a large multilingual machine translation model for translator behavior learning, characterized in that: include: Processor and memory; The memory is used to store computer programs; The processor is connected to the memory and is used to execute the computer program stored in the memory, so that the device for training a large multilingual machine translation model for translator behavior learning performs the training method for a large multilingual machine translation model for translator behavior learning as described in any one of claims 1 to 5.