Method and computer equipment for training large language model

Abstract

The invention discloses a method for training a large language model and computer equipment. Various embodiments relate to a method for training a Large Language Model (LLM) for an unseen task T ', comprising: obtaining a plurality of pre-trained Low Rank Adaptation (LoRA) modules; using the set of examples Q to obtain a set of weights for the plurality of pre-trained LoRA modules; using the set of weights on a plurality of pre-trained LoRA modules to obtain a fused LoRA module; the fused LoRA module is applied to the LLM to obtain an adapted LLM for the unseen task T '.

Description

Technical Field

[0001] Various aspects of the present disclosure relate to methods and computing devices for training large language models. Background Art

[0002] Recent advances in natural language processing (NLP) are mainly driven by large-scale pre-trained language models (LLMs), such as OpenAI GPT, Flan-T5, LLaMA, which achieve state-of-the-art performance on a wide range of NLP tasks. However, the large parameter size of these LLMs poses challenges in terms of computational efficiency and memory consumption during fine-tuning.

[0003] Therefore, efficient methods for natural language processing are needed. Summary of the invention

[0004] Various embodiments relate to a method for training a large language model (LLM) for an unseen task T', comprising: obtaining a plurality of pre-trained low-rank adaptation (LoRA) modules; using a set of examples Q to obtain a set of weights for the plurality of pre-trained LoRA modules; using the set of weights on the plurality of pre-trained LoRA modules to obtain a fused LoRA module; applying the fused LoRA module to the LLM to obtain an adapted LLM for the unseen task T'.

[0005] According to one embodiment, the adapted LLM is obtained by freezing the original weights of the LLM and introducing a low-rank matrix from the fused LoRA module that modifies a subset of the original weights of the LLM.

[0006] According to one embodiment, multiple pre-trained LoRA modules are trained using multiple upstream tasks T. The multiple upstream tasks T are N different types of upstream tasks for cross-task generalization.

[0007] According to one embodiment, the multiple pre-trained LoRA modules used to obtain the fused LoRA module have the same rank.

[0008] According to one embodiment, the set of weights is obtained by a gradient-free algorithm. The set of weights obtained by the gradient-free algorithm minimizes the cross entropy loss on the set of examples Q.

[0009] According to one embodiment, the set of examples Q is related to the unseen task T'. The number of examples Q may be a relatively small number, such as 5 examples.

[0010] Various embodiments relate to a computer device for training a large language model (LLM) for an unseen task T', comprising: a processor, a memory storing at least one program code, the at least one program code being loaded and executed by the processor to: obtain a plurality of pre-trained low-rank adaptation (LoRA) modules; use a set of examples Q to obtain a set of weights for a plurality of pre-trained LoRA modules; use the set of weights on a plurality of pre-trained LoRA modules to obtain a fused LoRA module; apply the fused LoRA module to the LLM to obtain an adapted LLM for an unseen task T'.

[0011] According to one embodiment, the adapted LLM is obtained by freezing the original weights of the LLM and introducing a low-rank matrix from the fused LoRA module that modifies a subset of the original weights of the LLM.

[0012] According to one embodiment, multiple pre-trained LoRA modules are trained using multiple upstream tasks T. The multiple upstream tasks T are N different types of upstream tasks for cross-task generalization.

[0013] According to one embodiment, the multiple pre-trained LoRA modules used to obtain the fused LoRA module have the same rank.

[0014] According to one embodiment, the set of weights is obtained by a gradient-free algorithm. The set of weights obtained by the gradient-free algorithm minimizes the cross entropy loss on the set of examples Q.

[0015] According to one embodiment, the set of examples Q is related to the unseen task T'. The number of examples Q may be a relatively small number, such as 5 examples.

[0016] According to one embodiment, a computer-readable storage medium is provided, wherein the storage medium stores at least one program code, and the at least one program code is executed by a processor to implement the above-mentioned training method.

[0017] According to one embodiment, a computer program product includes computer instructions stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device implements the above training method.

[0018] It should be noted that embodiments described in the context of a method of training a large language model are similarly valid for a computer device, and vice versa. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The invention will be better understood with reference to the detailed description when considered in conjunction with the non-limiting examples and accompanying drawings, in which:

[0020] - Figure 1A-Figure 1D Exemplary illustrations of various NLPs and their results are shown.

[0021] - Figure 2 An exemplary method for training a large language model according to an embodiment is shown.

[0022] - Figure 3 An exemplary diagram of the LoRA module stages (i.e., compose and adapt) according to an embodiment is shown.

[0023] - Figure 4A-4C Exemplary results of various NLPs according to embodiments are shown. DETAILED DESCRIPTION

[0024] The following specific embodiments refer to the accompanying drawings, which illustrate the specific details and embodiments in which the present disclosure can be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the present disclosure. Other embodiments may be utilized, and structural and logical changes may be made without departing from the scope of the present disclosure. Various embodiments are not necessarily mutually exclusive, as some embodiments may be combined with one or more other embodiments to form new embodiments.

[0025] Embodiments described in the context of one of the computer device or method are analogously valid for the other computer device or method. Similarly, embodiments described in the context of the computer device are analogously valid for the method, and vice versa.

[0026] Features described in the context of one embodiment may be correspondingly applicable to the same or similar features in other embodiments. Features described in the context of one embodiment may be correspondingly applicable to other embodiments, even if not explicitly described in these other embodiments. In addition, additions and / or combinations and / or replacements as described for features in the context of one embodiment may be correspondingly applicable to the same or similar features in other embodiments.

[0027] In the context of various embodiments, the articles "a", "an", and "the" used with respect to features or elements include reference to one or more features or elements.

[0028] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0029] Recent advances in natural language processing (NLP) are driven by large pre-trained language models (LLMs), and low-rank adaptation (LoRA) techniques have been proposed for effectively fine-tuning LLMs.

[0030] Previous work on LoRA mainly focused on its efficiency, but less attention was paid to the synthesizability of LoRA modules.

[0031] According to various embodiments, multiple LoRA modules can be grouped together to form LoRAHub, a strategic framework for synthesizing multiple LoRA modules trained for different seen tasks to obtain adaptable performance for unseen tasks. Using a few examples (e.g., 5 examples) from unseen tasks, LoRAHub can enable automatic synthesis of different LoRA modules without any human expertise. Experimental results on the Big-Bench Hard (BBH) benchmark show that LoRAHub can achieve performance close to contextual learning in a few-shot scenario without providing examples with each input during inference.

[0032] The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark designed to probe large language models and speculate on their future capabilities. Big-Bench Hard (BBH) is a subset of the challenge tasks from BIG-bench.

[0033] Figure 1A-Figure 1D Exemplary illustrations of various NLPs and their results are shown.

[0034] Figure 1A An exemplary illustration of a zero-shot learning LLM is shown.Zero-shot learning (ZSL) is a machine learning scenario in which an LLM is trained to recognize and classify objects or concepts without having seen any examples of these classes or concepts beforehand.

[0035] Figure 1B An exemplary diagram of few-shot context learning (LLM) is shown. Few-shot learning is a machine learning method that uses parameter adaptation to learn the best model parameters for a task with a limited number of supervised examples.

[0036] Figure 1C An exemplary diagram of few-shot LoRAHub learning according to an embodiment is shown.

[0037] Low Rank Adaptation (LoRA) is a parameter-efficient fine-tuning method to speed up the training of LLMs while significantly reducing memory requirements and computation. By freezing the parameters of the base model (e.g., LLM) and training pluggable LoRA modules, LoRA training tends to produce strong performance on downstream tasks.

[0038] Figure 1D Shows Figure 1A-Figure 1C Various NLP results.

[0039] The method disclosed herein ( Figure 1C ) achieves the same level of zero-shot learning ( Figure 1A ) similar inference throughput, and close to contextual learning on the BIG-Bench Hard (BBH) benchmark ( Figure 1B ) performance.

[0040] Figure 2 An exemplary method for training a large language model according to an embodiment is shown.

[0041] like Figure 2 As shown, there may be a method 200 for training a large language model. In the method 200, a first step 202 may include obtaining a plurality of pre-trained low rank adaptation (LoRA) modules. A second step 204 may include using a set of examples Q to obtain a set of weights for a plurality of pre-trained LoRA modules. A third step 206 may include using the set of weights on a plurality of pre-trained LoRA modules to obtain a fused LoRA module. A fourth step 208 may include applying the fused LoRA module to the LLM to obtain an adapted LLM for an unseen task T'.

[0042] It should be understood that the above Figure 2 The above operations are not limited to this specific order. Any suitable, modified order of operations can be used.

[0043] In various embodiments, a computer device for training a large language model (LLM) for an unseen task T' may include a processor for performing computing tasks (such as NLP and LLM training). The computer device may also include a memory. The memory may store at least one program code that is loaded and executed by the processor to train the large language model.

[0044] In an embodiment, the processor may obtain a plurality of pre-trained low rank adaptation (LoRA) modules.

[0045] In an embodiment, the processor may use the set of example Qs to obtain a set of weights for a plurality of pre-trained LoRA modules.

[0046] In an embodiment, the processor may use a set of weights on multiple pre-trained LoRA modules to obtain a fused LoRA module.

[0047] In an embodiment, the processor may apply the fused LoRA module to the LLM to obtain an adapted LLM for an unseen task T'.

[0048] According to one embodiment, the adapted LLM is obtained by freezing the original weights of the LLM and introducing a low-rank matrix from the fused LoRA module that modifies a subset of the original weights of the LLM.

[0049] According to one embodiment, multiple pre-trained LoRA modules are trained using multiple upstream tasks T. The multiple upstream tasks T are N different types of upstream tasks for cross-task generalization.

[0050] According to one embodiment, the multiple pre-trained LoRA modules used to obtain the fused LoRA module have the same rank.

[0051] According to one embodiment, the set of weights is obtained by a gradient-free algorithm. The set of weights obtained by the gradient-free algorithm minimizes the cross entropy loss on the set of examples Q.

[0052] According to one embodiment, the set of examples Q is related to the unseen task T'. The number of examples Q may be a relatively small number, such as 5 examples.

[0053] According to one embodiment, a computer-readable storage medium is provided, wherein the storage medium stores at least one program code, and the at least one program code is executed by a processor to implement the above-mentioned training method.

[0054] According to one embodiment, a computer program product includes computer instructions stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device implements the above training method.

[0055] In an embodiment, multiple LoRA modules can be used for task generalization. The technique goes beyond single-task training to strategically synthesize LoRA modules for adaptable performance on unseen tasks. This approach enables automatic LoRA module synthesis without relying on manual design or human expertise. With only a few examples (e.g., 5 examples) from unseen tasks, the method can automatically synthesize any compatible LoRA module without human intervention.

[0056] In other words, no a priori assumptions are made about which LoRA modules trained for which tasks can be synthesized, granting the flexibility to combine any modules as long as the modules meet the specification (e.g., use the same LLM).

[0057] In an embodiment, since the method utilizes several LoRA modules, it may be referred to as LoRAHub, and the method using the LoRAHub may be referred to as LoRAHub learning.

[0058] To validate the effectiveness of LoRAHub, experiments were conducted on the widely used BBH benchmark, where FLAN-T5 is as large as LLM. The experimental results demonstrate the effectiveness of synthesizing LoRA modules for unseen tasks through few-shot LoRAHub learning. For example, this approach achieved an average score of 34.7 over five runs, which is close to the performance of few-shot contextual learning (37.5). However, the inference cost of this approach is much lower than that of contextual learning because no examples need to be provided as input to LLM.

[0059] In addition, the learning procedure is computationally efficient because it uses a gradient-free method to adjust the synthesis coefficients of the LoRA module and only requires a few inference steps on unseen tasks. In experiments, with a single A100 card, few-shot LoRAHub learning on each BBH task takes less than 1 minute.

[0060] Finally, the development of LoRA modules allows users to share their trained LoRA modules so that others can use them for new tasks. This can grow into a library of reusable LoRA modules covering a wide variety of capabilities. It promotes collaborative AI development, allowing the community to jointly expand the capabilities of LLM through dynamic LoRA module synthesis. Sharing and reusing modules can maximize value across tasks, users, and computing resources.

[0061] In an embodiment, the large language model M θ It can be based on the Transformer architecture and can be pre-trained on a large-scale natural language corpus.

[0062] The model architecture can be encoder-decoder or decoder-only. θ It is also possible to fine-tune via a large set of instruction following datasets such as the FLAN Collection and PromptSource.

[0063] In an embodiment, any suitable large language model (LLM) may be used.

[0064] In an embodiment, the LoRA module can utilize cross-task generalization. Assume there are N different upstream tasks, referred to as T = {T 1 ,...,T N}. In real-world scenarios, it is common that users want specialized models to perform their own tasks (which the LLM has never seen before). For the target task T′ / ∈T, the user can only provide a small set of examples Q. The goal is to train the LLM M θAdaptation is performed to generalize to task T′ using only Q to achieve cross-task generalization.

[0065] Previously, a straightforward approach was to fine-tune M directly on Q. θ The weights of , in order to obtain an updated model M that performs better on T′ φ (like Figure 1B However, this can be inefficient, slow, and unstable when Q is small.

[0066] In an embodiment, LoRA-adjusted low-rank adaptation (LoRA) is a parameter-efficient fine-tuning method that can adapt the LLM using only a small external module, rather than fine-tuning the weights of the entire LLM. LoRA works by freezing the weights of the LLM and inserting a trainable low-rank factorization matrix as an adapter module in each layer. This module has far fewer trainable parameters than the full LLM, enabling fast and stable adaptation using limited examples. Therefore, LoRA provides an effective way to quickly personalize the LLM for new tasks using limited training data.

[0067] Figure 3 An exemplary diagram of the LoRA module stages (i.e. synthesis and adaptation) according to an embodiment is shown.

[0068] The LoRAHub approach consists of two phases: synthesis and adaptation. In the synthesis phase, the pre-trained LoRA modules are fused into a single module using a set of weights ω. In the adaptation phase, the fused LoRA module is evaluated on a few-shot example from an unseen task. The weights are then adapted to a new set of weights using a gradient-free algorithm. After performing N iterations, a well-adapted LoRA module is obtained.

[0069] like Figure 2 As shown, given N different upstream tasks, N specified upstream LoRA modules are trained respectively. For task T i ∈T, named m i Then, given a new unseen task T′ / ∈T, there exists a small set of examples Q. The set of examples Q is used to guide the synthesis of these LoRA modules.

[0070] In the synthesis phase, the set of weights {w1,w2,...,wN} is obtained via a gradient-free optimization algorithm. Then, the weights and upstream LoRA are used to obtain To get the fused LoRA module Make Better performance can be achieved on the few-sample example Q. Finally, the fused LoRA module is applied to LLM M θ to obtain the final adaptation as a well-adapted model for unseen tasks T′.

[0071] LoRA decomposes the attention weight update into a low-rank matrix to reduce the number of trainable parameters. In detail, in order to update the pre-trained weight matrix W0∈R d×k To impose constraints on the update, LoRA uses low-rank decomposition to represent W 0 +δW=W 0 +AB, where A∈R d×r , B∈R r×k And the rank r is set to be much smaller than d and k to reduce the number of trainable parameters. In an embodiment, m=AB represents a LoRA module.

[0072] An element-by-element synthesis method is used to combine LoRA modules. This technique requires adding parameters at corresponding positions to achieve synthesis. This requires that the LoRAs to be fused have the same rank r in order to have matching structures:

[0073]

[0074] Optimizing the weights for the LoRA module (e.g., hyperparameter search) poses challenges and requires additional memory space for gradient backpropagation.

[0075] In an embodiment, a gradient-free algorithm searches for suitable weights. Throughout the optimization process, a cross entropy loss is used to guide the procedure. The goal is to find a set of weights that minimizes the loss on the validation set Q.

[0076] In an embodiment, a combinatorial optimization method can be used, which is incorporated into a comprehensive algorithm set and selects appropriate optimization algorithms based on different scenarios. The subsequent experimental scenarios mainly utilize the covariance matrix adaptive evolution strategy (CMA-ES). CMA-ES is a stochastic optimization algorithm for solving continuous optimization problems. This algorithm is based on populations and does not rely on derivative information, making it suitable for a wide range of optimization tasks. CMA-ES operates by utilizing the covariance matrix to adapt the search distribution, thereby iteratively updating the mean and covariance of the distribution to optimize the objective function.

[0077] In an embodiment, LoRA reduces the number of trainable parameters by learning a rank factorization matrix pair while freezing the original weights. This greatly reduces the storage requirements of large language models adapted to specific tasks and enables efficient task switching during deployment without introducing inference delays. LoRA also outperforms several other adaptation methods, including adapters, prefix conditioning, and fine-tuning.

[0078] In the experiment, FLAN-T5 is selected as the base language model. FLAN-T5 is a series of language models with similar structures but different sizes. FLAN-T5 demonstrates excellent zero-shot and few-shot capabilities among models of corresponding sizes. The experiments are mainly focused on FLAN-T5 Large (780M parameters).

[0079] In the experiment, there is a set of trained LoRA. FLAN provides nearly 200 different tasks and their descriptions. On this basis, approximately 200 LoRA modules are trained as potential candidates. During each experimental process, 20 LoRA modules are randomly selected as candidates.

[0080] Experiments are conducted using the widely used BBH benchmark. BBH consists of a set of multiple-choice questions from a diverse set of problems. 27 tasks are used as challenging benchmarks for language models. In all tasks, exact match (EM) is used as the evaluation metric.

[0081] In the experiments, PEFT is used to implement LoRA, and rank r=16 is the default LoRA tuning hyperparameter. The gradient-free method is implemented by the open source gradient-free optimization library nevergrad. In this implementation, a constraint is imposed that the absolute value of the LoRA weight should not exceed 1.5, with an initial point of zero weight for all LoRA modules. In the default setting, the maximum step size is set to 40, which means that a maximum of 40 attempts can be made to calculate the loss on the sample, and only 5 examples are used during optimization. As for the hyperparameters used when training the candidate LoRA modules, the batch size, learning rate, and number of training epochs are fixed to 64, 1e, and 1e, respectively, for all candidate LoRA modules. -4 and 10.

[0082]

[0083]

[0084] Table 1

[0085] Table 1 shows the performance comparison between zero-shot learning (Zero), few-shot context learning (ICL), and the proposed few-shot LoRAHub learning (LoRAHub). The average performance of LoRAHub is calculated across 5 runs with different random seeds, and the best performance is reported as the maximum value over these runs for reference.

[0086] As shown in Table 1, on average, the proposed method performs comparable to in-context learning (ICL) across the 5-shot scenario. Notably, the proposed method has the same token usage as the zero-shot method, which is much less token count than ICL. In large language models, the length of the input is highly correlated with the cost when inferring. Our method exhibits some fluctuations, but in the vast majority of cases, it can outperform the performance of zero-shot. Moreover, in the best scenario, it can outperform ICL with fewer tokens.

[0087] 4A- Figure 4C Exemplary results of various NLPs according to various embodiments are shown.

[0088] Figures 4A-4C The performance of different methods on the aforementioned tasks at different sample sizes is shown. The x-axis represents the number of samples, while the y-axis represents the task performance, specifically measured using the exact match metric.

[0089] LoRAHub demonstrates strong few-shot learning capabilities. Both ICL and the methods disclosed in this paper are gradient-free methods, meaning they do not rely on gradient information for optimization. When comparing the methods disclosed in this paper with other gradient-based methods (such as back-propagation-based methods), the effectiveness can vary depending on the specific task and dataset.

[0090] Through the analysis of a binary classification task, a multiple choice question and a generation task selected from the BBH dataset. In the case of exemplary small sample sizes, all three methods show instability, as evidenced by the performance degradation as the number of samples increases. In this scenario, the method disclosed in this article demonstrates comparable or even better performance compared to the method using back propagation.

[0091] When given a larger sample size, only in relatively easy tasks, such as binary classification, the method disclosed herein shows comparable performance to gradient-based training methods. Although the method disclosed herein demonstrates significant improvements in multi-class classification and generation tasks, there is still a clear gap compared to conventional methods.

[0092]

[0093] Table 2

[0094] Table 2 presents the top five LoRAs identified as very useful. These five tasks mainly involve reading comprehension and reasoning, which are considered to be more demanding skills. Given that most of the selected tasks fall into the category of logical reasoning, there is a good reason to consider these tasks that are closely related to reasoning ability as the most important tasks. This also serves as evidence that our method effectively reorganizes the abilities within the candidate to better handle downstream tasks.

[0095] For the best LoRA that optimizes the final loss, an experiment is designed. Specifically, LoRAs associated with downstream tasks are incorporated into the LoRA candidate pool, and then the obtained results are evaluated to see whether they can approach or exceed the effectiveness demonstrated by individual LoRA instances.

[0096] LoRA trained on the task wikitablequestion(WTQ) is introduced to the existing LoRA candidates. These candidates were originally trained on the tasks included in the FLAN training set. Then, WTQ is defined as a downstream task and the LoRA weights of these LoRA candidates are found.

[0097] Among the final weight sets of all LoRA models, the LoRA model trained with the WTQ task exhibits the highest absolute weight value. In addition, the performance of the combined LoRA model slightly exceeds the individual performance of the WTQ LoRA model itself.

[0098] This successful integration confirms the effectiveness of our search method in identifying the best upstream LoRA model for downstream tasks. Moreover, it serves as empirical evidence of the promising results achieved by our approach.

[0099] The methods described herein may be performed, and the various processing or computing units and devices and computing entities described herein may be implemented, by one or more circuits. In an embodiment, a "circuit" may be understood as any kind of logic implementation entity, which may be hardware, software, firmware, or any combination thereof. Thus, in an embodiment, a "circuit" may be a hardwired logic circuit or a programmable logic circuit, such as a programmable processor, e.g., a microprocessor. A "circuit" may also be software implemented or executed by a processor, such as any kind of computer program, such as a computer program using virtual machine code. According to alternative embodiments, any other kind of implementation of the corresponding functions described herein may also be understood as a "circuit".

[0100] Although the present disclosure has been specifically shown and described with reference to specific embodiments, it will be understood by those skilled in the art that various changes in form and detail may be made to the present invention without departing from the spirit and scope of the present invention as defined in the appended claims. Therefore, the scope of the present invention is indicated by the appended claims, and therefore all changes within the equivalent meaning and scope of the claims are intended to be included.

Claims

1. A method for training a large language model (LLM) for an unseen task T', comprising: Obtaining multiple pre-trained low-rank adaptation modules, i.e., multiple pre-trained LoRA modules; Using the set of examples Q to obtain a set of weights for the plurality of pre-trained LoRA modules; Using the set of weights on the plurality of pre-trained LoRA modules to obtain a fused LoRA module; as well as The fused LoRA module is applied to the LLM to obtain an adapted LLM for the unseen task T'.

2. The method of claim 1, wherein the adapted LLM is obtained by freezing the original weights of the LLM and introducing a low-rank matrix from the fused LoRA module, the low-rank matrix modifying a subset of the original weights of the LLM.

3. The method according to claim 1 or 2, wherein a plurality of upstream tasks T are used to train the plurality of pre-trained LoRA modules. The method according to claim 3 , wherein the plurality of upstream tasks T are N different types of upstream tasks for cross-task generalization.

5. The method according to any one of claims 1 to 4, wherein the multiple pre-trained LoRA modules used to obtain the fused LoRA module have the same rank.

6. The method according to any one of claims 1 to 5, wherein the set of weights is obtained by a gradient-free algorithm.

7. A method according to claim 6, wherein the set of weights obtained by the gradient-free algorithm minimizes the cross-entropy loss over the set of examples Q.

8. The method according to any one of claims 1 to 7, wherein the set of examples Q is related to the unseen task T'.

9. The method of claim 8, wherein Q is 5.

10. A computer device for training a large language model (LLM) for an unseen task T', comprising: A processor, a memory, the memory storing at least one program code, the at least one program code being loaded and executed by the processor to: Obtaining multiple pre-trained low-rank adaptation modules, i.e., multiple pre-trained LoRA modules; Using the set of examples Q to obtain a set of weights for the plurality of pre-trained LoRA modules; Using the set of weights on the plurality of pre-trained LoRA modules to obtain a fused LoRA module; as well as The fused LoRA module is applied to the LLM to obtain an adapted LLM for the unseen task T'.

11. The computer device of claim 10, wherein the adapted LLM is obtained by freezing original weights of the LLM and introducing a low-rank matrix from the fused LoRA module, the low-rank matrix modifying a subset of the original weights of the LLM.

12. The computer device of claim 10 or 11, wherein the plurality of pre-trained LoRA modules are trained using a plurality of upstream tasks T. 13 . The computer device of claim 12 , wherein the plurality of upstream tasks T are N different types of upstream tasks for cross-task generalization.

14. The computer device according to any one of claims 10 to 13, wherein the plurality of pre-trained LoRA modules used to obtain the fused LoRA module have the same rank.

15. The computer device according to any one of claims 10 to 14, wherein the set of weights is obtained by a gradient-free algorithm.

16. The computer device of claim 15, wherein the set of weights obtained by the gradient-free algorithm minimizes a cross entropy loss over the set of examples Q.

17. Computer apparatus according to any one of claims 10 to 16, wherein the set of examples Q is related to the unseen task T'.

18. The computer device of claim 17, wherein Q is 5.

19. A computer-readable storage medium, characterized in that: The storage medium stores at least one program code, and the at least one program code is executed by a processor to implement the training method according to any one of claims 1 to 9.

20. A computer program product, comprising computer instructions stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to implement the training method according to any one of claims 1 to 9.

Images