Embedding model training method, system, electronic device and storage medium for LLM translation example selection

Abstract

The present invention discloses an Embedding model training method, system, electronic device and storage medium for LLM translation example selection. The method first selects positive and negative examples directly related to the LLM translation quality through a translation quality assessment model, and then retrains the Embedding model using these positive and negative examples, and adjusts the loss function during retraining, so that the translation examples obtained by retraining the Embedding model can significantly improve the translation quality of the LLM. Compared with the Embedding model trained by the traditional method, which only focuses on examples with semantics similar to the sentence to be translated, the Embedding model after retraining in the present invention focuses on those examples that can directly improve the translation quality of the sentence to be translated.

Description

Technical Field

[0001] The present invention belongs to the field of natural language processing in artificial intelligence, and particularly relates to an Embedding model training method, system, electronic device and storage medium for LLM translation example selection. Background Art

[0002] In-Context Learning (ICL) is a technique that integrates task demonstrations in natural language format into a prompt, and it is also an embodiment of the emerging capabilities of large language models (LLMs). This method allows pre-trained LLMs to solve new tasks without fine-tuning the model, only by providing task descriptions and corresponding examples. When using an LLM for natural language translation (taking Russian to Chinese as an example), the translation ability of the model can be stimulated only by constructing the following prompt and inputting it into the LLM:

[0003] prompt:

[0004] Please translate these English sentences into Chinese:

[0005] English: The patient was administered intravenous antibiotics.

[0006] Chinese: The patient received intravenous antibiotics.

[0007] English: Clinical trials demonstrate the efficacy of this new antiviral drug.

[0008] Chinese: Clinical trials show the effectiveness of this new antiviral drug.

[0009] English: Acute renal failure requires immediate dialysis.

[0010] Chinese: Acute renal failure requires immediate dialysis.

[0011] English: The biopsy confirmed the presence of malignant cells. (Sentence to be translated)

[0012] Chinese: (The large model will output the corresponding translation result)

[0013] That is, give the LLM a set of examples (input-output pairs), add a test input at the end of the examples, and allow the LLM to predict the output using only the conditioned examples.

[0014] In contextual learning, the choice of examples will affect the accuracy of LLM translation, so the choice of examples is crucial. Generally speaking, it is necessary to use a retrieval model to retrieve translation examples similar to the sentence to be translated in the dual corpus to construct a prompt. Figure 1 As shown in the figure, the most commonly used method is to encode the sentence to be translated into a vector through the Embedding model, and then perform a nearest neighbor search. The key lies in the use of the Embedding model, which is usually used for semantic matching, and its training task is to determine whether two sentences are related. Assuming that the sentence vector Sa, the positive example vector Sp and the negative example vector Sn are given, the training loss function of the Embedding model is usually: max(||Sa-Sp||-||Sa-Sn|| + ε,0). The purpose of this training loss is to make the vectors of sentences with similar semantics more closely represented in space after encoding. However, in the scenario of using traditional Embedding to retrieve examples, due to its training loss, the Embedding model will retrieve a large number of examples that are similar to the question semantics, which will cause redundancy in example information and affect the translation quality. Summary of the invention

[0015] In order to solve the above technical problems, the present invention provides an Embedding model training method, system, electronic device and storage medium for LLM translation example selection.

[0016] The technical solution provided by the present invention is as follows:

[0017] A method for training an Embedding model for LLM translation example selection includes the following steps:

[0018] S1. Encode the corpus through the pre-trained Embedding model and store the results in the vector database;

[0019] S2. Randomly extract a pair of translation sentences from the training set of the Embedding model, denoted as ( xq , yq ), xq Represents the source language text to be translated. yq Indicates the corresponding target language text;

[0020] S3. Use the pre-trained Emebdding model to xq Encode and retrieve from the vector database n Example translation,i The translation example is denoted as { x i , y i}, x i Indicates i The source language text in the translation example, y i Indicates i Target language text in the translation example;

[0021] S4. Respectively compare the retrieved translation examples with the source language text to be translated xq Splice them into prompts and send them to LLM for translation to obtain the translation results corresponding to each translation example;

[0022] S5. Use the translation quality assessment model to score each translation result;

[0023] S6. The source language text of the translation example corresponding to the translation result with the highest score or greater than the score threshold is taken as a positive example. xp , the source language texts of the translation examples corresponding to the remaining translation results are taken as negative examples, recorded as the negative example set { xn i},in xn i Indicates i Negative regulations;

[0024] S7, repeating steps S2 to S6 until a sufficient number of positive and negative examples are obtained;

[0025] S8. Use the obtained positive and negative examples to train the Embedding model.

[0026] Preferably, the translation quality assessment model in step S5 adopts the wmt22-comet-da model.

[0027] Furthermore, when the Embedding model is trained in step S8, the loss function used is:

[0028] ,

[0029] in, emb Represents the vector representation obtained by the Embedding model, d Represents the distance metric function.

[0030] Preferably, the distance metric function is Euclidean distance or cosine similarity.

[0031] An Embedding model training system based on the above method includes:

[0032] A bilingual corpus storing a plurality of sentence pairs, each of which is composed of a source language text and a target language text;

[0033] Vector database, used to store bilingual corpora encoded by pre-trained Embedding models;

[0034] An example retrieval module is used to retrieve translation examples related to the sentence to be translated from the vector database;

[0035] LLM translation module, used to translate the sentence to be translated according to the input prompt;

[0036] Translation scoring module, used to score the translation results;

[0037] A positive and negative example judgment module is used to judge the positive and negative examples of the translation examples obtained by the example retrieval module according to the scoring results of the translation scoring module;

[0038] The model retraining module is used to retrain the Embedding model using positive and negative examples.

[0039] An electronic device comprises a memory, a processor and a computer program stored and running on the memory, wherein the processor implements the steps of the above-mentioned Embedding model training method when executing the program.

[0040] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the processor executes the steps of the above-mentioned Embedding model training method.

[0041] Beneficial effects of the present invention:

[0042] The present invention first selects positive and negative examples directly related to the LLM translation quality through a translation quality assessment model, then uses these positive and negative examples to retrain the Embedding model, and adjusts the loss function during retraining, so that the translation examples obtained by retraining the Embedding model can significantly improve the translation quality of LLM. Compared with the Embedding model trained by the traditional method, which only focuses on examples with semantics similar to the sentence to be translated, the retrained Embedding model in the present invention focuses on those examples that can directly improve the translation quality of the sentence to be translated. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0044] Figure 1 is a schematic diagram of a method for obtaining translation examples provided by an embodiment of the present invention;

[0045] Figure 2 It is a schematic diagram of the Embedding model training process provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, other embodiments obtained by ordinary technicians in this field without making creative work are all within the scope of protection of the present invention.

[0047] Embodiment 1

[0048] like Figure 2 As shown, this embodiment provides an Embedding model training method for LLM translation example selection, taking English as the source language and Chinese as the target language as an example, the training method mainly includes the following steps:

[0049] Step 1: Encode the Chinese and English bilingual corpus through the pre-trained Embedding model and store the encoding results in the vector database.

[0050] Step 2: Randomly extract a pair of Chinese-English translation sentences from the Chinese-English bilingual training set of the Embedding model, denoted as ( xq , yq ), xq Indicates English, yq Indicates Chinese.

[0051] Step 3: Use the pre-trained Emebdding model to xq Encode and retrieve from the vector database via nearest neighbor search n Translation examples, of which i The translation example is denoted as { x i , y i}.

[0052] Step 4: Retrieve the n Translation examples and sentences to be translated xq Splice them into prompt, send them to LLM for translation, and get the translation result.

[0053] Step 5: For each translation result oi The translation quality assessment model is used for scoring. The translation quality assessment model used in this embodiment is wmt22-comet-da, which is a model for machine translation (MT) evaluation and is part of the Unbabel / COMET framework. The model is based on the XLM-R architecture and is fine-tuned by human judgment to generate translation quality assessments.

[0054] Step 6: Take the source language of the translation example corresponding to the translation result with the highest score or greater than a certain score threshold as the positive example xp , the source language of the translation examples corresponding to all other translation results are taken as negative examples, recorded as the set { xn i}, xn i Indicates i Negative exceptions.

[0055] Step 7: Loop through steps 2 to 6 until a sufficient amount of positive and negative examples are accumulated, then stop executing and use the accumulated positive and negative examples to train the Emebdding model according to the following training loss function:

[0056] ,

[0057] in, emb Represents the vector representation obtained by the embedding model, d represents a distance metric function, such as Euclidean distance or cosine similarity.

[0058] To verify the effectiveness of this method, a comparative test was conducted. The test LLM was the LLAMA-13B large model with 13 billion parameters. The test set was a multi-domain test set. The test fields included information technology (IT), law (LAW) and medical (Medical). The evaluation index was BLEU (BLEU was originally proposed by IBM in 2002 and was used to evaluate the performance of statistical machine translation systems). The final test results are shown in Table 1. As can be seen from Table 1, the translation quality of the LLAMA-13B large model has been effectively improved through the method of this embodiment, and its BLEU scores in various fields have been improved to varying degrees.

[0059] Table 1 Comparative test results

[0060]

[0061] Embodiment 2

[0062] This embodiment provides an Embedding model training system based on the above method, including:

[0063] A bilingual corpus storing a plurality of sentence pairs, each of which is composed of a source language text and a target language text;

[0064] Vector database, used to store bilingual corpora encoded by pre-trained Embedding models;

[0065] An example retrieval module is used to retrieve translation examples related to the sentence to be translated from the vector database;

[0066] LLM translation module, used to translate the sentence to be translated according to the input prompt;

[0067] Translation scoring module, used to score the translation results;

[0068] A positive and negative example judgment module is used to judge the positive and negative examples of the translation examples obtained by the example retrieval module according to the scoring results of the translation scoring module;

[0069] The model retraining module is used to retrain the Embedding model using positive and negative examples.

[0070] Embodiment 3

[0071] This embodiment provides an electronic device, which may include a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus. The processor may call logic instructions in the memory to execute the steps of the above-mentioned Embedding model training method.

[0072] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in several computer-readable storage media when sold or used as independent products. Based on such an understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of any of the methods described in the first embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0073] The above-mentioned product can execute the Embedding model training method described in Example 1, and has the corresponding functional modules and beneficial effects of the method. For technical details not described in detail in this embodiment, please refer to the Embedding model training method provided in Example 1 of the present invention.

[0074] Embodiment 4

[0075] This embodiment provides a computer-readable storage medium, the type of which is as described in Embodiment 3, and the computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the processor executes the steps of the Embedding model training method described in Embodiment 1.

[0076] The above-mentioned product can execute the Embedding model training method described in Example 1, and has the corresponding functional modules and beneficial effects of the method. For technical details not described in detail in this embodiment, please refer to the Embedding model training method provided in Example 1 of the present invention.

[0077] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution, in essence or in other words, the part that contributes to the relevant technology, can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0078] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Under the concept of the present invention, the technical features in the above embodiments or different embodiments may also be combined, the steps may be implemented in any order, and there are many other changes in different aspects of the present invention as described above, which are not provided in detail for the sake of simplicity. Although the present invention has been described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. An Embedding model training method for LLM translation example selection, characterized in that: The following steps are involved: S1. Encode the corpus through the pre-trained Embedding model and store the results in the vector database; S2. Randomly extract a pair of translation sentences from the training set of the Embedding model, denoted as ( xq , yq ), xq Represents the source language text to be translated. yq Indicates the corresponding target language text; S3, use the pre-trained Emebdding model to xq Encode and retrieve from the vector database n Example translation, i The translation example is denoted as { x i , y i }, x i Indicates i The source language text in the translation example, y i Indicates i Target language text in the translation example; S4. Respectively compare the retrieved translation examples with the source language text to be translated xq Splice them into prompts and send them to LLM for translation to obtain the translation results corresponding to each translation example; S5. Use the translation quality assessment model to score each translation result; S6. The source language text of the translation example corresponding to the translation result with the highest score or greater than the score threshold is taken as a positive example. xp , the source language texts of the translation examples corresponding to the remaining translation results are taken as negative examples, recorded as the negative example set { xn i },in xn i Indicates i Negative regulations; S7, repeating steps S2 to S6 until a sufficient number of positive and negative examples are obtained; S8. Use the obtained positive and negative examples to train the Embedding model.

2. The Embedding model training method according to claim 1, characterized in that: The translation quality assessment model in step S5 adopts the wmt22-comet-da model.

3. The Embedding model training method according to claim 1, characterized in that: When the Embedding model is trained using positive and negative examples in step S8, the loss function used is: ， in, emb Represents the vector representation obtained by the Embedding model, d Represents the distance metric function.

4. The Embedding model training method according to claim 3, characterized in that: The distance measurement function is Euclidean distance or cosine similarity.

5. An Embedding model training system based on the method according to any one of claims 1 to 4, characterized in that: include: A bilingual corpus storing a plurality of sentence pairs, each of which is composed of a source language text and a target language text; Vector database, used to store bilingual corpora encoded by pre-trained Embedding models; An example retrieval module is used to retrieve translation examples related to the sentence to be translated from the vector database; LLM translation module, used to translate the sentence to be translated according to the input prompt; Translation scoring module, used to score the translation results; A positive and negative example judgment module is used to judge the positive and negative examples of the translation examples obtained by the example retrieval module according to the scoring results of the translation scoring module; The model retraining module is used to retrain the Embedding model using positive and negative examples.

6. An electronic device comprising a memory, a processor and a computer program stored and running on the memory, characterized in that: When the processor executes the program, the steps of the Embedding model training method described in any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the processor executes the steps of the Embedding model training method as described in any one of claims 1 to 4.

Images