A pre-inference parallel reasoning method, device and medium for machine translation of large language models

Abstract

The present invention belongs to the technical field of machine translation, and provides a pre-inference parallel reasoning method, device and medium for a machine-translated large language model, wherein the method comprises: (1) pre-inference token #imgabs0#: run the machine-translated large language model, perform a non-autoregressive form of reasoning, and the machine-translated large language model randomly guesses the token of each position in the translation y according to the input original text x, which is recorded as #imgabs1#; (2) perform parallel translation: record the current iteration number as t, perform a parallel machine translation according to the pre-inference #imgabs2# in step (1), or the parallel result #imgabs3# in the previous step, calculate and obtain a further token of each translation position, which is recorded as the parallel result #imgabs4#; (3) verify and modify the results of the parallel translation; (4) cache the existing results. The present invention solves the problem of insufficient parallelism in the traditional large language model reasoning process through pre-inference and parallel translation, and optimizes the potential errors caused by the parallel translation process.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine translation, and in particular, relates to a pre-inference parallel reasoning method, device and medium for a large language model for machine translation. Background Art

[0002] With the rise of large language models such as chatGPT, applying large language model technology to the field of machine translation has become a hot topic in academia. However, in machine translation practice, the reasoning speed of large language models for machine translation is very low. Some recent studies have shown that the reasoning speed of large language models may be two orders of magnitude slower than traditional machine translation models, which has become a problem in promoting large language models for machine translation.

[0003] The above problems are caused by two aspects: on the one hand, the number of parameters of the large language model is too large, resulting in excessive computational complexity during the inference process. Even for a lightweight version of the large language model, the number of parameters is tens to hundreds of times that of the traditional machine translation model. On the other hand, the autoregressive reasoning method cannot be implemented in parallel. Similar to the traditional machine translation model, the large language model still uses autoregressive decoding for reasoning. In this reasoning method, the model can only predict each token serially and cannot be processed in parallel, which greatly increases the time cost and reduces the utilization rate of the graphics card. Summary of the invention

[0004] The purpose of the present invention is to provide a pre-inference parallel reasoning method for machine-translated large language models to solve the technical problems existing in the prior art.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0006] A pre-inference parallel reasoning method for machine translation of a large language model includes the following steps:

[0007] (1) Pre-inference token :Run the machine translation language model and perform a non-autoregressive reasoning. The machine translation language model takes the input original text x, the first token of the machine translation text is recorded as s, the translation length is n, and randomly guesses the token at each position in the translation y, recorded as :

[0008] ;

[0009] (2) Perform parallel translation: Let the current number of iterations be t, and according to the pre-inference in step (1) , or the parallel result of the previous step , perform a parallel machine translation, calculate the further token of each translation position, and record it as the parallel result ;

[0010] (3) Verify and modify the results of parallel translation: Based on the parallel results of the previous step of the current step or the result of initialization , and the parallel result of the current step Perform token-by-token comparison. If the i-th token in the current step The calculated word list probability is greater than the i-th token in the previous step If the probability is , the reasoning result is updated, otherwise, the reasoning result update is stopped, and the previous reasoning result is copied to the reasoning result of the current step to obtain a preliminary reasoning result;

[0011] (4) Cache of existing results: The preliminary inference results obtained in step (3) are segmented into n-grams and cached into the candidate pool, which is recorded as , and then perform parallel reasoning to obtain the final iterative result, which is recorded as:

[0012] .

[0013] Preferably, in step (2), when parallel machine translation is performed for the first time, the parallel results The calculation method is:

[0014] .

[0015] Preferably, in step (2), when executing parallel machine translation for subsequent time steps, the parallel results The calculation method is:

[0016] .

[0017] Preferably, in step (3), when the parallel machine translation is performed for the first time in step (2), the initialization result is , and the parallel result of the current step Perform a token-by-token comparison.

[0018] Preferably, in step (3), when the parallel machine translation in step (2) is performed for a subsequent time step, the parallel result of the previous step of the parallel translation of the current step is used. , and the parallel result of the current step Perform a token-by-token comparison.

[0019] In order to achieve the above-mentioned object, the present invention also provides a computer-readable storage medium on which a computer program is stored, and the computer program is executed by a processor to implement the pre-inference parallel reasoning method for machine-translated large language models.

[0020] In order to achieve the above-mentioned purpose, the present invention also provides a device for predictive parallel reasoning for machine-translated large language models, comprising: a processor and a memory; the memory is used to store a computer program; the processor is connected to the memory, and is used to execute the computer program stored in the memory, so that the device for predictive parallel reasoning for machine-translated large language models executes the predictive parallel reasoning method for machine-translated large language models.

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] (1) The present invention solves the problem of insufficient parallelism in the traditional large language model reasoning process through pre-inference and parallel translation.

[0023] (2) The present invention optimizes potential errors caused by the parallel translation process through result verification and modification.

[0024] (3) The present invention makes full use of the historical translation results of the large language model of machine translation through the n-gram cache pool, thereby reducing the number of running rounds of the parallel translation process. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a schematic diagram of the principle of Example 1 of the present invention.

[0026] Figure 2 This is a principle block diagram of Example 3 of the present invention. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to have a clearer understanding and knowledge of the present invention, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described below are only used to explain the present invention, and are convenient for understanding. The technical solutions provided by the present invention are not limited to the technical solutions provided by the following embodiments, and the technical solutions provided by the embodiments should not limit the protection scope of the present invention.

[0028] It should be noted that the illustrations provided in the following embodiments are only used to illustrate the basic concept of the present invention in a schematic manner. Therefore, the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the form, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0029] Example 1

[0030] like Figure 1As shown, this embodiment provides a pre-inference parallel reasoning method for machine-translated large language models, which can improve the reasoning parallelism of machine-translated large language models.

[0031] In the reasoning process of the traditional machine translation large language model, assuming that the machine translation original text is x, the i-th token in the machine translation text is , the first token of the machine-translated text is denoted as s, and the machine-translated language model is denoted as function , then the autoregressive formal reasoning satisfies the following system of equations:

[0032]

[0033] In the serial reasoning process, if you want to calculate , then you must get .

[0034] But if in all It was known before , the above equations can be calculated in parallel, that is, Perform pre-inference. Assume that the i-1th pre-inference token is , then the above equations can be rewritten as:

[0035]

[0036] In order to achieve parallel reasoning and high-quality translation, this embodiment provides a specific technical solution, which is as follows:

[0037] 1. Pre-inference token

[0038] Run the machine translation large language model and perform a non-autoregressive reasoning (non-autoregressive reasoning is synchronous reasoning, that is, all tokens are inferred simultaneously in a form that does not rely on context; autoregressive reasoning is asynchronous reasoning, that is, serial reasoning, which requires waiting for the previous result to be generated before the current result is generated). The machine translation large language model randomly guesses the token at each position in the translation y based on the input original text x, which is recorded as :

[0039] ;

[0040] 2. Perform parallel translation:

[0041] Let the current number of iterations be t, according to the pre-inference , or the parallel result of the previous step , perform a parallel machine translation, calculate the further token of each translation position, and record it as the parallel result ; where the previous step refers to the previous time step. If the current step is the second time step, the previous step is a non-autoregressive result, that is, ; If the current step is a subsequent time step (the third and subsequent time steps), the previous step is the previous step in the iteration process.

[0042] In this step, the parallel results The calculation method of is based on the above two different situations, which can be divided into two specific types:

[0043] One is that when parallel machine translation is first performed, the parallel results The calculation method is:

[0044] ;

[0045] Second, when performing parallel machine translation for subsequent time steps, the parallel results The calculation method is:

[0046] .

[0047] 3. Verify and modify the results of parallel translation

[0048] Parallel results of the previous step based on the parallel translation of the current step or the result of initialization , and the parallel result of the current step Perform token-by-token comparison. In this step, the parallel result of the previous step of the parallel translation of the current step is compared. or the result of initialization The selection method is as follows: (1) When the parallel machine translation is performed for the first time in step 2, the initialization result , and the parallel result of the current step Token-by-token comparison; (2) When the parallel machine translation in step 2 is performed for the subsequent time step, the parallel result of the previous step of the parallel translation of the current step is used to compare the tokens. , and the parallel result of the current step Perform a token-by-token comparison.

[0049] The comparison method is as follows: If the i-th token of the current step The calculated word list probability is greater than the i-th token in the previous step If the probability is , the inference result is updated and the iteration is continued. Otherwise, the inference result update and iteration are stopped, and the previous inference result is copied to the inference result of the current step to obtain a preliminary inference result.

[0050] 4. Existing result cache

[0051] The preliminary inference results are segmented into n-grams and cached into the candidate pool, which is recorded as , in the next iteration, using replace , and then perform parallel reasoning again, the new iteration result can be recorded as:

[0052]

[0053] The technical solution provided in this embodiment solves the problem of insufficient parallelism in the traditional large language model reasoning process through pre-inference and parallel translation, and optimizes potential errors caused by the parallel translation process.

[0054] Example 2

[0055] This embodiment provides a computer-readable storage medium, on which a computer program is stored, and the computer program is executed by a processor to implement the pre-inference parallel reasoning method for machine-translated large language models provided in Embodiment 1. A person of ordinary skill in the art can understand that all or part of the steps of implementing the method provided in Embodiment 1 can be completed by hardware related to the computer program, and the above-mentioned computer program can be stored in a computer-readable storage medium, and when the program is executed, the steps of the method provided in Embodiment 1 are executed; and the above-mentioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes.

[0056] Example 3

[0057] like Figure 2 As shown, this embodiment provides a pre-inference parallel reasoning device for machine-translated large language models, including: a processor and a memory; the memory is used to store computer programs; the processor is connected to the memory, and is used to execute the computer programs stored in the memory, so that the pre-inference parallel reasoning device for machine-translated large language models executes the pre-inference parallel reasoning method for machine-translated large language models provided in Example 1.

[0058] Specifically, the memory includes: ROM, RAM, disk, USB flash drive, memory card or CD and other media that can store program codes.

[0059] Preferably, the processor can be a general-purpose processor, including a central processing unit, a network processor, etc.; it can also be a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component.

[0060] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A pre-inference parallel reasoning method for machine-translated large language models, characterized in that: The following steps are involved: (1) Pre-inference of token S :Run the machine translation language model and perform a non-autoregressive form of reasoning. The machine translation language model takes the input original text x, the first token of the machine translated text is recorded as s, the length of the translation is n, and randomly guesses the token at each position in the translation y, recorded as y s : (2) Perform parallel translation: Let the current iteration number be t, and according to the pre-inference y in step (1) s , or the parallel result of the previous step Perform a parallel machine translation and calculate the further token of each translation position, which is recorded as the parallel result y t ; (3) Verify and modify the results of parallel translation: Based on the parallel results of the previous step of the current step Or the result of initialization y s , and the parallel result y of the current step t Perform token-by-token comparison. If the i-th token in the current step The calculated word list probability is greater than the i-th token in the previous step If the probability is , the reasoning result is updated, otherwise, the reasoning result update is stopped, and the previous reasoning result is copied to the reasoning result of the current step to obtain a preliminary reasoning result; (4) Cache existing results: The preliminary inference results obtained in step (3) are segmented into n-grams and cached in the candidate pool, which is recorded as y c , and then perform parallel reasoning to obtain the final iterative result, which is recorded as: and={and (t-1) ,and c ,and t }; In step (2), when parallel machine translation is performed for the first time, the parallel result y t The calculation method is: In step (2), when executing parallel machine translation for subsequent time steps, the parallel result y t The calculation method is: In step (3), when the parallel machine translation is performed for the first time in step (2), the initialization result y s , and the parallel result y of the current step t Perform token-by-token comparison; In step (3), when the parallel machine translation in step (2) is performed for a subsequent time step, the parallel result y of the previous step of the parallel translation of the current step is calculated. t-1 , and the parallel result y of the current step t Perform a token-by-token comparison.

2. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer program is executed by a processor to implement the pre-inference parallel reasoning method for machine translation of a large language model as claimed in claim 1.

3. A device for predictive parallel reasoning of large language models for machine translation, characterized in that: include: Processor and memory; The memory is used to store computer programs; The processor is connected to the memory and is used to execute the computer program stored in the memory, so that the device for predictive parallel reasoning for machine-translated large language models executes the predictive parallel reasoning method for machine-translated large language models as claimed in claim 1.

Images