A multilingual machine translation low-rank adaptation sharing strategy learning method and system

Abstract

The application discloses a multilingual machine translation low-rank adaptive sharing strategy learning method and system, relates to the technical field of multilingual neural machine translation, and comprises the following steps: constructing a LoRA architecture in a large language model; defining a language gate vector and performing normalization to obtain a probability vector; generating a mask through Bernoulli sampling based on the probability vector; introducing the mask between two linear transformations of the LoRA architecture to calculate a final output result '; performing approximate gradient calculation through a pass-through estimator to obtain the gradient of the probability vector; training and reasoning the large language model; evaluating the multilingual translation performance of the large language model; dynamically learning and optimizing a cross-language sharing strategy of the LoRA rank, effectively relieving language knowledge interference, and improving the translation performance of the model.

Description

Technical Field

[0001] This invention relates to the field of multilingual neural machine translation technology, and in particular to a method and system for learning low-rank adaptation sharing strategies in multilingual machine translation. Background Technology

[0002] Multilingual machine translation aims to achieve efficient translation between multiple languages ​​through a unified model, overcoming the cascading errors of traditional serial translation.

[0003] In existing technologies, the main approach is to use a Transformer-based architecture to perform multilingual joint training by sharing encoder-decoder, adding language-specific modules, or introducing low-rank adaptation.

[0004] First, complete parameter sharing can lead to knowledge conflicts between languages, causing negative interference and reducing the translation performance of some language pairs. Second, while setting up a LoRA module independently for each language can alleviate interference, it isolates the parameters between languages ​​and makes it impossible to achieve knowledge transfer between related languages. Finally, existing methods have difficulty dynamically and adaptively selecting the rank within LoRA during training, making it impossible to achieve optimal parameter sharing between some languages.

[0005] Therefore, a method and system for learning low-rank adaptation sharing strategies in multilingual machine translation are provided to solve the above problems. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for learning low-rank adaptation sharing strategies in multilingual machine translation, enabling automatic learning of LoRA rank cross-language sharing and uniquely set strategies, and effectively improving the performance of multilingual translation models.

[0007] To achieve the above objectives, this invention provides a method for learning low-rank adaptation sharing strategies in multilingual machine translation, comprising the following steps: S1: Building a LoRA architecture within a large language model; S2: Define language-gated vectors Then normalize to obtain the probability vector. ; S3: Based on probability vectors A mask is generated through Bernoulli sampling. ; S4: Introducing a mask between two linear transformations in the LoRA architecture Selectively enable LoRA rank for a specific language and compute the final output of LoRA. '; S5: Approximate gradient calculation is performed using a pass-through estimator to obtain the probability vector. gradient ; S6: Training and inference of large language models; S7: Evaluate the multilingual translation performance of large language models.

[0008] Preferably, step S1 specifically involves assuming that the parameter update matrix of the large language model has an intrinsic rank, and using low-rank decomposition technology to transform the parameter matrix of the pre-trained model into two trainable low-rank matrices, ultimately outputting the result. Specifically set as follows: ; in, This represents the parameter matrix of the pre-trained model. This indicates that the target parameter is updated. This represents the input feature vector of a large language model. This represents a low-rank matrix initialized by a Gaussian distribution. This represents a low-rank matrix initialized to 0. Represents the set of real numbers. The parameter matrix representing the pre-trained model Dimensions This indicates the introduction of a low-rank. Represents the input feature vector Batch size, Indicates sentence length.

[0009] Preferably, step S2 specifically involves processing each target language... Define the gate vector Gating vector The dimension is ,pass The activation function will gate the vector The values ​​of each dimension are normalized to the interval [0,1] to obtain the probability vector. probability vector The larger the value of the median dimension, the more important the dimension is to the target language. The greater the importance of a dimension, the greater the probability that that dimension will be sampled during the sampling process; the probability vector... Specifically set as follows: .

[0010] Preferably, in step S3, the mask For a binary vector, the positions sampled as 1 represent the rank involved in updating parameters during forward propagation, and the mask. Specifically set as follows: .

[0011] Preferably, step S4 specifically includes the following steps: S41: Obtain the low-rank matrix of the LoRA architecture. Output results low-rank matrix Output results Specifically set as follows: ; S42: Mask With low-rank matrix Output results Perform a dot product to obtain the output result of the dot product. The output of the dot product Specifically set as follows: ; S43: Output the dot product result Input to low-rank matrix Based on the hidden states generated by selectively enabling LoRA rank for a specific language, the final output is obtained. Final output result Specifically set as follows: .

[0012] Preferably, step S5 specifically includes the following steps: S51: The sampling process approximates the gradient using a pass-through estimator, where the mask... Set the gradient of the non-differentiable nodes to 1 and perform backpropagation. The chain rule is specifically set as follows: ; in, Indicates the differentiation operation; S52: Calculate the probability vector gradient probability vector gradient Specifically set as follows: ; in, express The gradient.

[0013] Preferably, step S6 specifically includes the following steps: S61: During the training phase, the mask is dynamically selected based on the target language. Update the rank of the activated LoRA and the corresponding gate vector. ; S62: Input the prompt template, source sentence and target sentence into the large language model, calculate the cross-entropy loss between the prediction results output by the large language model and the target sentence, obtain the difference between the prediction distribution and the true label distribution, and train the large language model through teacher-forced method; S63: During the inference phase, load the model parameters of the pre-trained model and the low-rank adapted model, input the prompt template and source statement, autoregressively decode to generate the target statement, and output until the end label is predicted or the maximum generation length set by the model is reached. S64: Select LoRA ranks with an activation probability greater than 0.5 by using a fixed threshold of 0.5 to translate the target statement.

[0014] Preferably, step S7 specifically involves: training a large language model on multiple language pairs, calculating its performance using BLEU scores, and comparing it with baseline methods to evaluate the translation performance of the large language model.

[0015] A system for learning a low-rank adaptation sharing strategy in multilingual machine translation includes: Multilingual parallel corpus acquisition module: used to collect multilingual data, combine prompt templates to construct a dataset, and perform word segmentation according to the basic large language model to construct a data format that can be used for training; Basic Large Language Model Module: This module is used to construct the basic model framework of the system, and all subsequent modules are trained on this basis. LoRA computation module: used to develop efficient parameter fine-tuning methods, combined with the basic large language model module for fine-tuning training; Language mask sampling module: used to enable language-dependent LoRA rank; The pass-through estimator gradient approximation module is used to set the backpropagation chain rule and compute the gradient for LoRA parameter updates. Parameter update module: Used for LoRA parameter updates during model training.

[0016] Therefore, the present invention employs the above-mentioned method and system for learning low-rank adaptation sharing strategies in multilingual machine translation, which has the following beneficial effects: (1) This scheme uses language gating vectors and Bernoulli sampling mechanism to dynamically select a subset of LoRA rank for each target language, thereby achieving adaptive sharing of cross-language parameters. This not only avoids the negative interference caused by full parameter sharing, but also achieves knowledge transfer between languages ​​by sharing partial rank, thus improving the overall performance of multilingual translation. (2) This scheme uses Bernoulli hard sampling to replace soft gating, which prompts the model to learn sparse language-specific representations, reduces the linear dependence between language features, and combines a pass-through estimator to approximate gradients, solves the problem of non-differentiability of sampling, ensures effective propagation of gradients during training, and improves optimization stability and generalization ability. (3) This scheme is based on low-rank adaptation technology. It can achieve multilingual adaptation by training only a small number of parameters, which greatly reduces the memory usage and computational overhead. Experiments show that this method can stably improve the translation effect on a variety of large models such as Gemma, Llama, and Qianwen, and has good cross-model applicability and scalability.

[0017] The method of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0018] Figure 1 This is a flowchart of a method and system for learning low-rank adaptation sharing strategies in multilingual machine translation according to the present invention. Figure 2 For the application of LoRA in this invention Masking Flowchart for fine-tuning; Figure 3 This is a flowchart of the training and inference process of a large language model according to the present invention; Figure 4 This is a schematic diagram illustrating the training objective of the autoregressive model of this invention; Figure 5 This is a system architecture diagram of a low-rank adaptation sharing strategy learning method and system for multilingual machine translation according to the present invention. Detailed Implementation

[0019] The method of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0020] Unless otherwise defined, the methodological or scientific terms used in this invention shall have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains.

[0021] The terms "comprising" or "including" as used in this invention mean that the element preceding the term encompasses the element listed after the term, and do not exclude the possibility of encompassing other elements. Terms such as "inner," "outer," "upper," and "lower" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. When the absolute position of the described object changes, the relative positional relationship may also change accordingly. In this invention, unless otherwise explicitly specified and limited, the term "attached" and similar terms should be interpreted broadly. For example, it can refer to a fixed connection, a detachable connection, or an integral part; it can refer to a direct connection or an indirect connection through an intermediate medium; it can refer to the internal communication of two elements or the interaction relationship between two elements. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0022] Example like Figures 1-4 As shown, this invention provides a low-rank adaptation sharing strategy learning method for multilingual machine translation, comprising the following steps: S1: Building a LoRA architecture within a large language model; Step S1 specifically assumes that the parameter update matrix of the large language model has an intrinsic rank. The parameter matrix of the pre-trained model is converted into two trainable low-rank matrices by low-rank decomposition technology, thereby reducing the memory usage caused by excessive parameter updates and bringing about an approximate performance improvement.

[0023] Final output results Specifically set as follows: ; in, This represents the parameter matrix of the pre-trained model. This represents the target update parameters. Size by Turn to , This represents the input feature vector of a large language model. This represents a low-rank matrix initialized by a Gaussian distribution. This represents a low-rank matrix initialized to 0; only low-rank matrices exist. and low-rank matrix It will be trained and updated, and the parameter matrix of the pre-trained model It has been frozen.

[0024] Represents the set of real numbers. The parameter matrix representing the pre-trained model Dimensions Indicates the introduced low rank, the introduced low rank The parameter matrix is ​​much smaller than that of the pre-trained model. Dimensions , Represents the input feature vector Batch size, Indicates sentence length.

[0025] S2: Define language-gated vectors Then normalize to obtain the probability vector. ; Step S2 specifically involves, to obtain the importance of each dimension to the language, performing the following steps for each target language: Define the gate vector Gating vector Quantity and target language The number is the same, and the gate vector is... The dimension is , and the introduced low-rank Same, through The activation function will gate the vector The values ​​of each dimension are normalized to the interval [0,1] to obtain the probability vector. probability vector The larger the value of the median dimension, the more important the dimension is to the target language. The greater the importance of a dimension, the greater the probability that that dimension will be sampled during the sampling process; the probability vector... Specifically set as follows: .

[0026] S3: Based on probability vectors A mask is generated through Bernoulli sampling. ; In step S3, the mask For a binary vector, the positions sampled as 1 represent the rank involved in updating parameters during forward propagation, and the mask. Specifically set as follows: .

[0027] The application of rank-sharing policy learning in LoRA includes three steps: (1) The pre-trained model consists of N layers stacked together. Each layer consists of two modules: a self-attention mechanism and a feedforward neural network. Operations such as residuals, normalization, and activation functions are omitted. The self-attention mechanism includes query (Q), key (K), value (V), and output vector (O), which are used to calculate the relationship between the current token and other tokens. Feedforward neural networks capture more feature information through dimensionality increase and then dimensionality reduction operations. In this embodiment, Gemma, Llama and Qianwen large language models are used. They already have strong general text generation capabilities. In order to further improve their capabilities in downstream tasks and considering the cost of fine-tuning, an efficient fine-tuning method that only fine-tunes some parameters is usually adopted.

[0028] (2) The LoRA architecture can be applied to any linear transformation in the pre-trained model to update some parameters of the pre-trained model to adapt to the downstream task, where the mask This is the application location in LoRA.

[0029] (3) By selectively utilizing the LoRA rank of a specific language through masking operations, different dimensions are selected for forward propagation and gradient update for different target languages. Unlike direct LoRA fine-tuning, all languages ​​propagate and update all dimensions.

[0030] S4: Introducing a mask between two linear transformations in the LoRA architecture Selectively enable LoRA rank for a specific language and compute the final output of LoRA. '; Step S4 specifically includes the following steps: S41: Obtain the low-rank matrix of the LoRA architecture. Output results low-rank matrix Output results Specifically set as follows: ; S42: Mask With low-rank matrix Output results Perform a dot product to obtain the output result of the dot product. The output of the dot product Specifically set as follows: ; S43: Output the dot product result Input to low-rank matrix Based on the hidden states generated by selectively enabling LoRA rank for a specific language, the final output is obtained. Final output result Specifically set as follows: .

[0031] During fine-tuning, only the LoRA rank associated with that language is used to reduce interference between different languages. At the same time, different languages ​​that activate the same LoRA rank achieve mutual knowledge transfer through the shared LoRA rank.

[0032] Bernoulli sampling is non-differentiable; it determines whether the current rank is selected based on probability.

[0033] The reason why Bernoulli sampling is chosen instead of continuous probability values ​​for soft gating in this embodiment is as follows: (1) Hard sampling promotes the sparsity of language-specific representations, forcing the gating to make binary decisions on each dimension at each training step, which can effectively prevent the model from simply linearly weighting all language features; (2) The randomness introduced by Bernoulli sampling can be regarded as a regularization with uncertainty, because the language to which a text belongs is not always certain. Hard sampling training process makes the model not overly dependent on a certain absolutely stable language signal, so as to enhance its generalization ability.

[0034] S5: Approximate gradient calculation is performed using a pass-through estimator to obtain the probability vector. gradient ; Step S5 specifically includes the following steps: S51: The method of approximating gradient calculation using a pass-through estimator preserves non-differentiable operations during forward propagation. The sampling process approximates the gradient using a pass-through estimator, where the mask... Set the gradient of the non-differentiable nodes to 1 and perform backpropagation. The chain rule is specifically set as follows: ; in, Indicates the differentiation operation; S52: Calculate the probability vector gradient probability vector gradient Specifically set as follows: ; in, express The gradient.

[0035] S6: [s] and [e] are the start and end labels of the pre-trained model, respectively, for training and inference of the large language model; Step S6 specifically includes the following steps: S61: During the training phase, the mask is dynamically selected based on the target language. Update the rank of the activated LoRA and the corresponding gate vector. ; S62: Input the prompt template, source sentence, and target sentence into the large language model. Calculate the cross-entropy loss between the predicted results and the target sentence output by the large language model to obtain the difference between the predicted distribution and the true label distribution. Train the large language model using a teacher-forced method. The training objective is the next token prediction task; that is, at each time step, its input is the previous token of the true target sequence, i.e., y. n Instead of the model predicting the generated y in the previous time step, n ', y1 to y n And the end label [e] is for each token for which the loss needs to be calculated, while each time step during inference is generated autoregressively, i.e., y n =y n '; S63: During the inference phase, load the model parameters of the pre-trained model and the low-rank adapted model, input the prompt template and source statement, autoregressively decode to generate the target statement, and output until the end label is predicted or the maximum generation length set by the model is reached. S64: Select LoRA ranks with an activation probability greater than 0.5 by using a fixed threshold of 0.5 to translate the target statement.

[0036] S7: Evaluate the multilingual translation performance of large language models.

[0037] Step S7 involves training a large language model on multiple language pairs, calculating its performance using BLEU scores, and comparing it with baseline methods to evaluate the translation performance of the large language model.

[0038] Table 1: Public Corpus Directions and Data Size in the WMT18 News Field

[0039] As shown in Table 1, this embodiment uses publicly available corpora from the WMT18 news domain. The table only shows the data size for the English to other language direction (En-xx), while the size of the other language to English direction (xx-En) is equal to that of En-xx.

[0040] 200k sentence pairs were randomly selected for each translation direction as the training set. Except for the English-Anisha direction, which used newsdev2018 as the validation set, all other language directions used newtest2017 as the validation set. The newtest2018 test set was used for model performance evaluation.

[0041] The prompt template used during instruction fine-tuning is Translatefrom[SRC]to[TGT], where [SRC] and [TGT] represent the language tags of the source and target statements, respectively.

[0042] During the experiment, the number of fine-tuning cycles was set to 3, and the parameters were updated once the target words accumulated to approximately 17k. The Adam optimizer was used, and the learning rate was fixed at 1e-4. In this embodiment, the sacreBLEU toolkit was used to implement BLEU evaluation. The model with the highest accuracy on the validation set was used for result evaluation, and a beam search decoding was performed using a beam size of 4. All models used the same prompt template in the experiment.

[0043] Table 2: Experimental results of different methods

[0044] As shown in Table 2, this embodiment compares with existing research on the Gemma pre-trained model. Each column in the table represents each translation direction. The experimental results of this invention are denoted as RSSL. This embodiment outperforms all the baseline models compared on average. Specifically, compared with the strongest baseline MLSA-LoRA, it has a performance improvement of 5.99 in average BLEU value.

[0045] Table 3: Experimental results of applying the method of this embodiment to different pre-trained models

[0046] As shown in Table 3, this embodiment also compares the results of direct prompts and LoRA fine-tuning for different pre-trained models. Compared with the results of direct LoRA fine-tuning, the Llama and Qianwen large models using the method of this embodiment improve the average BLEU by 0.47 and 0.48, respectively.

[0047] like Figure 5 As shown, the present invention also provides a system for learning a low-rank adaptation sharing strategy in multilingual machine translation, comprising: Multilingual parallel corpus acquisition module: used to collect multilingual data, combine prompt templates to construct a dataset, and perform word segmentation according to the basic large language model to construct a data format that can be used for training; Basic Large Language Model Module: This module is used to construct the basic model framework of the system, and all subsequent modules are trained on this basis. LoRA computation module: used to develop efficient parameter fine-tuning methods, combined with the basic large language model module for fine-tuning training; Language mask sampling module: used to enable language-dependent LoRA rank; The pass-through estimator gradient approximation module is used to set the backpropagation chain rule and compute the gradient for LoRA parameter updates. Parameter update module: Used for LoRA parameter updates during model training.

[0048] This system can be used for multilingual translation. For models with parameter sizes of 8B and below, it can be deployed on single-GPU machines with more than 20GB of video memory, such as RTX4090, to accelerate inference and complete translation.

[0049] Therefore, the present invention adopts the above-mentioned method and system for learning low-rank adaptation sharing strategies in multilingual machine translation. By dynamically learning low-rank adaptation sharing strategies for each language during the LoRA fine-tuning process, it can automatically explore the sub-network of a specific language and solve the interference problem that exists when different languages ​​share parameters completely.

[0050] Finally, it should be noted that the above embodiments are only used to illustrate the method of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the method of the present invention, and these modifications or equivalent substitutions should not cause the modified method to deviate from the spirit and scope of the method of the present invention.

Claims

1. A low-rank adaptation sharing strategy learning method for multilingual machine translation, characterized in that, Includes the following steps: S1: Building a LoRA architecture within a large language model; S2: Define language-gated vectors Then normalize to obtain the probability vector. ; S3: Based on probability vectors A mask is generated through Bernoulli sampling. ; S4: Introducing a mask between two linear transformations in the LoRA architecture Selectively enable LoRA rank for a specific language and compute the final output of LoRA. '; S5: Approximate gradient calculation is performed using a pass-through estimator to obtain the probability vector. gradient ; S6: Training and inference of large language models; S7: Evaluate the multilingual translation performance of large language models.

2. The method for learning low-rank adaptation sharing strategies in multilingual machine translation according to claim 1, characterized in that, Step S1 specifically involves assuming that the parameter update matrix of the large language model has an intrinsic rank. It then uses low-rank decomposition techniques to transform the parameter matrix of the pre-trained model into two trainable low-rank matrices, ultimately outputting the result. Specifically set as follows: ； in, This represents the parameter matrix of the pre-trained model. This indicates that the target parameter is updated. This represents the input feature vector of a large language model. This represents a low-rank matrix initialized by a Gaussian distribution. This represents a low-rank matrix initialized to 0. Represents the set of real numbers. The parameter matrix representing the pre-trained model Dimensions This indicates the introduction of a low-rank. Represents the input feature vector Batch size, Indicates sentence length.

3. The method for learning low-rank adaptation sharing strategies in multilingual machine translation according to claim 2, characterized in that, Step S2 specifically involves processing each target language... Define the gate vector Gating vector The dimension is ,pass The activation function will gate the vector The values ​​of each dimension are normalized to the interval [0,1] to obtain the probability vector. probability vector The larger the value of the median dimension, the more important the dimension is to the target language. The greater the importance of a dimension, the greater the probability that that dimension will be sampled during the sampling process; the probability vector... Specifically set as follows: 。 4. The method for learning low-rank adaptation sharing strategies in multilingual machine translation according to claim 3, characterized in that, In step S3, the mask For a binary vector, the positions sampled as 1 represent the rank involved in updating parameters during forward propagation, and the mask. Specifically set as follows: 。 5. The method for learning low-rank adaptation sharing strategies in multilingual machine translation according to claim 4, characterized in that, Step S4 specifically includes the following steps: S41: Obtain the low-rank matrix of the LoRA architecture. Output results low-rank matrix Output results Specifically set as follows: ； S42: Mask With low-rank matrix Output results Perform a dot product to obtain the output result of the dot product. The output of the dot product Specifically set as follows: ； S43: Output the dot product result Input to low-rank matrix Based on the hidden states generated by selectively enabling LoRA rank for a specific language, the final output is obtained. Final output result Specifically set as follows: 。 6. The method for learning low-rank adaptation sharing strategies in multilingual machine translation according to claim 5, characterized in that, Step S5 specifically includes the following steps: S51: The sampling process approximates the gradient using a pass-through estimator, where the mask... Set the gradient of the non-differentiable nodes to 1 and perform backpropagation. The chain rule is specifically set as follows: ； in, Indicates the differentiation operation; S52: Calculate the probability vector gradient probability vector gradient Specifically set as follows: ； in, express The gradient.

7. The method for learning low-rank adaptation sharing strategies in multilingual machine translation according to claim 6, characterized in that, Step S6 specifically includes the following steps: S61: During the training phase, the mask is dynamically selected based on the target language. Update the rank of the activated LoRA and the corresponding gate vector. ; S62: Input the prompt template, source sentence and target sentence into the large language model, calculate the cross-entropy loss between the prediction results output by the large language model and the target sentence, obtain the difference between the prediction distribution and the true label distribution, and train the large language model through teacher-forced method; S63: During the inference phase, load the model parameters of the pre-trained model and the low-rank adapted model, input the prompt template and source statement, autoregressively decode to generate the target statement, and output until the end label is predicted or the maximum generation length set by the model is reached. S64: Select LoRA ranks with an activation probability greater than 0.5 by using a fixed threshold of 0.5 to translate the target statement.

8. The method for learning low-rank adaptation sharing strategies in multilingual machine translation according to claim 7, characterized in that, Step S7 specifically involves training a large language model on multiple language pairs, calculating its performance using BLEU scores, and comparing it with baseline methods to evaluate the translation performance of the large language model.

9. A system applying the low-rank adaptation sharing strategy learning method for multilingual machine translation according to any one of claims 1-8, characterized in that, include: Multilingual parallel corpus acquisition module: used to collect multilingual data, combine prompt templates to construct a dataset, and perform word segmentation according to the basic large language model to construct a data format that can be used for training; Basic Large Language Model Module: Used to form the basic model framework of the system, and subsequent modules are all trained on this basis; LoRA computation module: used to develop efficient parameter fine-tuning methods, combined with the basic large language model module for fine-tuning training; Language mask sampling module: used to enable language-dependent LoRA rank; The pass-through estimator gradient approximation module is used to set the backpropagation chain rule and compute the gradient for LoRA parameter updates. Parameter update module: Used for LoRA parameter updates during model training.

Images