Large model fine tuning method, system and device based on LoRA-MoE and medium

Abstract

The invention relates to the technical field of artificial intelligence, in particular to a large model fine tuning method, system and device based on LoRA-MoE and a medium. According to the method, the characteristic that the LoRA technology can use smaller parameters to carry out fine adjustment on a large model is combined with the characteristic that the MoE architecture can enable the model to be higher in generalization ability and better in effect, and the LoRA-MoE technology is adopted to carry out fine adjustment on the large model; according to the method, the problems that a large number of video memories need to be consumed during fine adjustment of the large model and the effect is poor when fine adjustment is conducted through the LoRA technology can be solved, it is ensured that consumption can be reduced in the fine adjustment process of the large model, and a certain fine adjustment effect is ensured.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a large model fine-tuning method, system, device and medium based on LoRA-MoE. Background Art

[0002] In recent years, large model technology has become a very hot research field in the field of artificial intelligence. Especially since OpenAI released chatGPT, this large model based on large computing power and large-scale language data as training samples has shown good performance in various fields. It can not only generate corresponding text, pictures, videos, etc. according to user needs, but also can be trained on specific data sets according to the specific needs of users to achieve better performance to handle specific downstream tasks. However, although the substantial increase in the parameters of the large model has brought about a huge improvement in performance, it also means that if a large model is to be trained from scratch, it often requires a large amount of training data and expensive training equipment, and it will also consume a lot of training time and huge computing resources. Therefore, it is very meaningful to study how to fine-tune the large model under the condition of limited computing resources and make it fit well with downstream tasks, and promote the application of large models in vertical fields. In order to achieve this goal, a more "lightweight" large model transfer learning technology called Parameter-Efficient Fine-Tuning (PEFT) is usually used, which includes efficient parameter fine-tuning methods such as adapter, prefix-Tuning, and low-rank adaptation (LoRA), which can fine-tune large models with limited computing resources and achieve certain downstream task training effects. The Mixtures of Local Experts (MoE) model was first proposed in 1991. The study proposed a new supervised learning process, that is, a system contains multiple separate networks, each network to process a subset of all training samples. For example, if there are some natural subsets in a data set (such as different topics), then if a single model is used to learn, it will be susceptible to a lot of interference, which will cause the model to have problems such as slow learning speed and difficulty in normalization. The hybrid expert model can use multiple models to learn and use a gating network to decide which model should be used to train each data, reducing the interference between different samples. Each such model is also called an expert.

[0003] Low-Rank Adaptation (LoRA): is a method commonly used for fine-tuning large models. Its characteristic is that it can fine-tune a small number of parameters so that the large model can achieve better performance in solving specific downstream tasks. Studies have found that because the model is over-parameterized, it has a smaller intrinsic dimension, and the large model mainly relies on this lower intrinsic dimension to complete the task adaptation. Therefore, in the process of task adaptation, even if the parameters are projected to a smaller subspace, the large model can still learn effectively. Therefore, LoRA adds a bypass next to the original pre-trained model, replacing the parameters that need to be trained with a dimensionality reduction matrix A and a dimensionality increase matrix B, and simulating the so-called intrinsic rank.

[0004] However, the current LoRA technology still has certain defects in the fine-tuning effect of large models, and there is currently a lack of fine-tuning methods that combine LoRA technology and MoE technology. Summary of the invention

[0005] In view of the problems that a large amount of video memory is consumed when fine-tuning a large model and the fine-tuning effect is poor when using the LoRA technology, the present invention proposes a large model fine-tuning method, system, device and medium based on LoRA-MoE; the method combines the characteristic of LoRA technology that can use smaller parameters to fine-tune a large model with the characteristic of MoE architecture that can make the model more generalized and have better effects, and proposes to use LoRA-MoE technology to fine-tune the large model. This method can solve the problem that a large amount of video memory is consumed when fine-tuning a large model and the problem that the fine-tuning effect is poor when using the LoRA technology, ensuring that the large model can reduce consumption during the fine-tuning process and ensure a certain fine-tuning effect.

[0006] The specific implementation contents of the present invention are as follows:

[0007] A large model fine-tuning method based on LoRA-MoE specifically includes the following steps:

[0008] Step S1: Obtain a data set and a pre-trained large model for natural language processing tasks;

[0009] Step S2: construct multiple LoRA adapters as MoE experts next to multiple linear layers of the large model;

[0010] Step S3: In the j-layer t module of the large model, establish an expert routing S jt Specify different LoRA adapters for input token data x;

[0011] Step S4: According to the low-rank matrix size r in the selected LoRA adapter and the parameters to be fine-tuned, freeze the reason weights of the large model and update the LoRA adapter parameters to obtain the fine-tuned large model for the natural language processing task.

[0012] In order to better implement the present invention, further, the specific operation of step S2 is: in the weight matrix of the j-layer t module of the large model Next to N j For a low-rank matrix A jt , low-rank matrix B jt As N j N adapters j MoE experts.

[0013] According to a large model fine-tuning method based on LoRA-MoE according to claim 2, it is characterized in that step S2 also includes: adjusting the number of LoRA adapters according to different layers of the large model, so that the number of experts in the high layer is greater than the number of experts in the low layer.

[0014] In order to better implement the present invention, further, the low-rank matrix A jt Initialized according to a random Gaussian distribution, the low-rank matrix B jt Initialized to 0, satisfying Ensure that the rank of the low-rank matrix is ​​much smaller than the rank of the weight matrix r<<min(d q ,d p ).

[0015] In order to better implement the present invention, further, the specific operation of step S3 is: establish a trainable expert weight matrix W jt , calculate the gated score F of the MOE expert according to the set TOPK parameters and the input oken data x i jt .

[0016] In order to better implement the present invention, further, the step S4 specifically includes the following steps:

[0017] Step S41: Based on the calculated gating score F i jt , construct the loss function L lbc And apply it to the load balancing loss of each layer of the large model;

[0018] Step S42: The result h obtained by the j-layer t module of the large model jt It can be expressed as the input x and the weight of this layer of the large model The product of the expert routing S jt From N jThe first K MoE experts with the highest gate control scores selected by the MoE experts are K pairs of low-rank matrices The product of the input x and its routing weight h jt ;

[0019] Step S43: According to the set hyperparameters, product h jt , loss function L lbc , and get the final loss function L total .

[0020] Based on the above-mentioned LoRA-MoE-based large model fine-tuning method, in order to better realize the present invention, further, a LoRA-MoE-based large model fine-tuning system is proposed, which is used to execute the above-mentioned LoRA-MoE-based large model fine-tuning method; characterized in that it includes a collection unit, a construction unit, a designation unit, and a fine-tuning unit;

[0021] The acquisition unit is used to obtain data sets and pre-trained large models for natural language processing tasks;

[0022] The construction unit is used to construct multiple LoRA adapters as MoE experts beside multiple linear layers of the large model;

[0023] The designation unit is used to establish an expert routing S in the j-layer t module of the large model. jt Specify different LoRA adapters for input token data x;

[0024] The fine-tuning unit is used to freeze the large model cause weights and update the LoRA adapter parameters according to the low-rank matrix size r in the selected LoRA adapter and the parameters to be fine-tuned, so as to obtain a fine-tuned large model for the natural language processing task.

[0025] Based on the above-mentioned LoRA-MoE-based large model fine-tuning method, in order to better realize the present invention, further, an electronic device is proposed, including a memory and a processor; a computer program is stored on the memory; when the computer program is executed on the processor, the above-mentioned LoRA-MoE-based large model fine-tuning method is implemented.

[0026] Based on the above-mentioned LoRA-MoE-based large model fine-tuning method, in order to better realize the present invention, further, a computer-readable storage medium is proposed, on which computer instructions are stored; when the computer instructions are executed on the above-mentioned electronic device, the above-mentioned LoRA-MoE-based large model fine-tuning method is implemented.

[0027] The present invention has the following beneficial effects:

[0028] (1) The LoRA fine-tuning method based on the MoE architecture of the present invention can achieve fine-tuning performance on unseen data sets that exceeds the performance of fine-tuning using only LoRA under the condition of updating a small amount of model parameters. At the same time, when using LoRA fine-tuning, if the instruction fine-tuning data for the model is too large, it will destroy the world knowledge stored in the large model, causing the performance of the large model to significantly decrease in some tests that test the world knowledge of the large model. The LoRA-MoE fine-tuning method can, to a certain extent, slow down the forgetting of world knowledge when fine-tuning the large model, and can show excellent model performance in specific downstream tasks. The model performs better in solving the corresponding downstream tasks and has less world knowledge forgetting.

[0029] (2) Compared with full-scale fine-tuning, the LoRA fine-tuning of the present invention greatly reduces the number of model fine-tuning parameters, and the overall number of parameters has dropped by more than ten times, which also greatly reduces the memory usage and training time during model training. At the same time, because LoRA adds the "residual" gradient on the basis of the full gradient of the main model, it saves the process of updating the weights of the main model optimizer, and the amount of calculation does not change significantly.

[0030] (3) In the analysis and research of MoE in the present invention, the use of some experts may be redundant, which may easily lead to overfitting of the gating network. Therefore, in order to solve the problem of how many LoRA adapters are used as experts in each layer of the large model, more LoRA experts are added to the high-level layers of the large model and fewer LoRA experts are added to the low-level layers to alleviate the problem of expert redundancy. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a schematic diagram of the LoRA-MoE architecture provided by the present invention.

[0032] Figure 2 This is a schematic diagram of the number of model layers provided by the present invention and the corresponding LoRA expert distribution.

[0033] Figure 3 This is a schematic diagram of the code summarization task results provided by the present invention.

[0034] Figure 4 A schematic diagram of the code defect detection task results provided by the present invention. DETAILED DESCRIPTION

[0035] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. It should be understood that the described embodiments are only part of the embodiments of the present invention, not all of the embodiments, and therefore should not be regarded as limiting the scope of protection. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technical personnel in this field without making creative work are within the scope of protection of the present invention.

[0036] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "disposed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0037] Embodiment 1:

[0038] This embodiment proposes a large model fine-tuning method based on LoRA-MoE, which specifically includes the following steps:

[0039] Step S1: Obtain the data set and pre-trained large model for natural language processing tasks.

[0040] Step S2: Build multiple LoRA adapters as MoE experts beside multiple linear layers of the large model.

[0041] The specific operation of step S2 is: in the weight matrix of the j-layer t module of the large model Next to N j For a low-rank matrix A jt , low-rank matrix B jt As N j N adapters j MoE experts.

[0042] The step S2 further includes: adjusting the number of LoRA adapters according to different layers of the large model, so that the number of experts in the high layer is greater than the number of experts in the low layer.

[0043] Furthermore, the low-rank matrix A jt Initialized according to a random Gaussian distribution, the low-rank matrix B jt Initialized to 0, satisfying Ensure that the rank of the low-rank matrix is ​​much smaller than the rank of the weight matrix r<<min(d q ,d p ).

[0044] Step S3: In the j-layer t module of the large model, establish an expert routing S jt Specify a different LoRA adapter for the input token data x.

[0045] The specific operation of step S3 is: establish a trainable expert weight matrix W jt , calculate the gated score F of the MOE expert according to the set TOPK parameters and the input oken data x i jt .

[0046] Step S4: According to the low-rank matrix size r in the selected LoRA adapter and the parameters to be fine-tuned, freeze the reason weights of the large model and update the LoRA adapter parameters to obtain the fine-tuned large model for the natural language processing task.

[0047] Furthermore, the step S4 specifically includes the following steps:

[0048] Step S41: Based on the calculated gating score F i jt , construct the loss function L lbc And apply it to the load balancing loss of each layer of the large model;

[0049] Step S42: The result h obtained by the j-layer t module of the large model jt It can be expressed as the input x and the weight of this layer of the large model The product of the expert routing S jt From N j The first K MoE experts with the highest gate control scores selected by the MoE experts are K pairs of low-rank matrices The product of the input x and its routing weight h jt ;

[0050] Step S43: According to the set hyperparameters, product h jt , loss function L lbc , and get the final loss function L total .

[0051] Working principle: This embodiment combines the characteristic of LoRA technology that can use smaller parameters to fine-tune large models with the characteristic of MoE architecture that can make the model more generalized and have better effects, and proposes to use LoRA-MoE technology to fine-tune large models. This method can solve the problem of consuming a large amount of video memory when fine-tuning large models and the problem of poor effect when fine-tuning using LoRA technology, ensuring that large models can reduce consumption during the fine-tuning process and ensure a certain fine-tuning effect.

[0052] Embodiment 2:

[0053] This embodiment is based on the above embodiment 1. Figure 1 , Figure 2 , Figure 3 , Figure 4 As shown, a specific embodiment is described in detail, which specifically includes the following steps:

[0054] Step S1: Given a dataset for a natural language processing-related task and a pre-trained large model;

[0055] Step S2: construct multiple LoRA adapters as MoE experts next to multiple linear layers in the large model, and make the number of high-level experts in the large model larger and the number of low-level experts smaller;

[0056] Step S3: Establish a route S jt Specify different LoRA adapters for input token data x;

[0057] Step S4: Select the size r of the low-rank matrix in the appropriate LoRA adapter and the relevant parameters that need to be fine-tuned, and finally obtain the fine-tuned large model for natural language processing tasks by freezing the original weights of the large model and updating the LoRA adapter parameters.

[0058] like Figure 1 As shown, Figure 1 The snowflake in the pre-trained parameter part indicates that the weight matrix is ​​frozen during training and will not be changed. The flame in the low-rank matrix indicates that the weight matrix in this part will change during training.

[0059] In specific implementation, the step S1 includes the following specific steps given a data set for a natural language processing-related task and a pre-trained large model:

[0060] The main choice is the large model of the decode-only architecture mode, and you can choose a targeted pre-trained large model according to the specific task type. For example, when processing code-related tasks, you can use codellama, starcoder and other code-related large models.

[0061] In specific implementation, the step S2 of constructing the LoRA adapter includes the following specific steps:

[0062] The weight matrix of the j-layer t module of the large model Next to N j For low-rank matrices As N j LoRA adapters, that is, N j MoE experts;

[0063] Low rank matrix Ajt Initialized according to a random Gaussian distribution, and the low-rank matrix B jt Initialize to 0, and make the low-rank matrix A jt and the low-rank matrix B jt satisfy r<<min(d q ,d p );

[0064] The number of LoRA adapters is adjusted according to different layers of the large model. The Llama3-8B used in the experiment has a total of 32 decoder layers. Two LoRA adapters are set as MoE experts in the first 8 layers, and 4 LoRA adapters are set as MoE experts in the next 8 layers. Six LoRA adapters are set as MoE experts in the next 8 layers, and 8 LoRA adapters are set as MoE experts in the last 8 layers to ensure that the number of MoE experts in the high-level of the large model is large, and the number of MoE experts in the low-level is small.

[0065] In the specific implementation, in step S3, a routing It can calculate the corresponding gating score for the input token data x And according to the score, different LoRA adapters are specified. The specific steps are as follows:

[0066] In routing Contains a trainable expert weight matrix Used to obtain the initial gating score, K is set to 2, that is, each time the expert router calculates the gating score of the MoE expert corresponding to each layer, it will select the two MoE experts with the highest scores to process the token data x. At this time, after inputting the token data x, the gating score of the i-th MoE expert calculated by the expert router is for:

[0067]

[0068] in The initial gated scores of each MoE expert and the input data x are obtained, and then the relative probabilities are obtained by normalizing Softmax. After that, the TopK operation will retain the first K largest values ​​and set the remaining values ​​to negative infinity. Finally, the relative probability corresponding to the i-th MoE expert is obtained. And finally get the corresponding gating score It can be seen that the first K MoE experts with the highest gating scores will retain the corresponding gating scores, and the gating scores of the remaining MoE experts will be negative infinity, and the expert routing S jt The top K MoE experts will be selected for calculation based on the result.

[0069] And apply load balancing loss on each layer To ensure fairness in selecting experts:

[0070]

[0071] At this time, during the inference process, the result h obtained by the j-layer t module of the large model jt It can be expressed as the input x and the weight of this layer of the large model The product of the expert routing Select K experts, that is, K pairs of low-rank matrices The product of the input x and its routing weight:

[0072]

[0073] h jt It is the final result obtained by the model under this method, that is, the required result obtained by using the model.

[0074] The final loss function is the original loss of LoRA The expert load balancing loss under this method is:

[0075]

[0076] Among them, β is a hyperparameter used to control the constraint strength of local balance.

[0077] This embodiment mainly combines the LoRA technology with the MoE technology, and adopts a design in which the number of high-level experts in the model is relatively large and the number of low-level experts is relatively small, so that a certain fine-tuning effect can be guaranteed while fine-tuning a large model using relatively few computing resources. At the same time, it also alleviates the problem of expert redundancy in MoE, and can use a more reasonable expert allocation method, which reduces the number of experts to a certain extent and improves the efficiency of model fine-tuning.

[0078] Step S4: Implement model fine-tuning technology. Adjust the corresponding hyperparameters according to the specific computing resources, where lora_r is used to determine the size of the low-rank matrix in LoRA. Generally speaking, the larger the lora_r parameter, the higher the degree of low-rank decomposition, and the number of model parameters will be further reduced, but it may lead to a decrease in model performance; and the smaller the lora_r parameter, the lower the degree of low-rank decomposition will be, the number of model parameters will increase, but the performance of the model can be improved. At the same time, the lora_alpha parameter defines the learning rate scaling factor adapted by LoRA, which affects the update speed of the low-rank matrix. Under this parameter, LoRA will introduce an additional expansion coefficient scaling in the forward propagation, and its value is: It can be seen that the larger the lora_alpha is, the greater the impact of the LoRA weight is. In this embodiment, the lora_alpha value is set to twice that of lora_r.

[0079] Experiments and analysis:

[0080] The experiment used the code summary and code defect detection datasets provided by CodeXGLUE. In the code summary task, this experiment selected Java as the main language for the code summary task. The code summary data selected by CodeXGLUE is mainly based on the data in CodeSearchNet. In the further processing of the data, examples that cannot be parsed into the Abstract Syntax Tree (AST) in the document are deleted, and examples where the document is not in English are deleted. The code defect detection data provided by CodeXGLUE comes from a paper on vulnerability detection by neural networks. It evaluates manually labeled functions in 4 large and small C language open source projects (Linux Kernel, QEMU, Wireshark, FFmpeg) and collects security-related submission information. The dataset is divided into training set, validation set, and test set in a ratio of 80%, 10%, and 10%.

[0081] The pre-trained model used in the experiment is Llama3-8B. Llama3 is a large model of the Decode-Only type designed based on Transformer. Its model architecture mainly includes an embedding layer (Embedding), a decoder layer (LlamaDecoderLayer), an output layer normalization (norm), and a language model head (lm_head). The model contains 32 decoder layers, each of which contains a self-attention layer (self\_attn), a multi-layer perceptron (mlp), and two layer normalizations. The self-attention layer contains four linear projection layers: q_proj, k_proj, v_proj, and o_proj, which are used to calculate the query vector Q, the key vector K, the value vector V, and the final projection for the output, respectively. RoPE position encoding is used to preserve the position information in the input so that the model can better understand the contextual meaning. The multi-layer perceptron in the decoder contains a gated projection (gate_proj), an ascending projection (up_proj), a descending projection (down_proj), and an activation function (act_fn) using SiLUA to allow small negative numbers to pass, thereby retaining more information.

[0082] When using the LoRA-MoE fine-tuning technology, we first add multiple pairs of low-rank matrices, namely adapters, to the model according to the traditional LoRA fine-tuning method. The experiment chooses to set corresponding LoRA adapters next to the linear layers such as q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, and down_proj in each decoder of the model. At the same time, according to the MoE architecture, we add multiple LoRA adapters as experts to each corresponding layer of the model. In the first 8 decoder layers of the Llama3-8B model, 2 LoRA adapters are set as experts for each linear layer, 4 LoRA adapters are set as experts for the next 8 layers, 6 LoRA adapters are set as experts for the next 8 layers, and 8 LoRA adapters are set as experts for the last eight layers. At the same time, the Top K of each layer is set to 2, that is, each time the expert routing calculates the gating score of the LoRA expert in each layer for the data, the two LoRA experts with the highest scores are selected to process the data.

[0083] In order to evaluate and compare the performance of the model after fine-tuning, this embodiment involves three baseline models: CodeLLama, CodeGeex, and ChatGPT. These models are selected to explore their performance on the code rule repair task at different temperatures T, especially the performance changes after special fine-tuning.

[0084] In terms of evaluation effect, the BLUE-4 method was used to test the model's generation results for code summaries. BLEU (Bilingual Evaluation Understudy) is a bilingual evaluation understudy. It first calculates the accuracy based on n-gram;

[0085]

[0086] After that, the brevity penalty factor BP is calculated based on the sentence length c generated by the model and the reference sentence length r;

[0087]

[0088] Finally, calculate the final BLUE score:

[0089] The final experiment of the model showed that the fine-tuning method using LoRA-MoE can achieve better results by training a small number of parameters while using less computing resources. The BLUE score obtained by using the LoRA-MoE method on the code summarization task was 20.17, while the BLUE score obtained by fine-tuning the model using only the LoRA method was 19.17; the LoRA-MoE method had an accuracy rate of 66.52% on the defect detection task, which was higher than the accuracy rate of 65.64 using only LoRA fine-tuning.

[0090] The other parts of this embodiment are the same as those of the above-mentioned embodiment 1, and thus will not be described in detail.

[0091] Embodiment 3:

[0092] Based on any one of the above-mentioned embodiments 1-2, this embodiment proposes a large model fine-tuning system based on LoRA-MoE, which is used to execute the above-mentioned large model fine-tuning method based on LoRA-MoE; it is characterized by comprising a collection unit, a construction unit, a designation unit, and a fine-tuning unit;

[0093] The acquisition unit is used to obtain data sets and pre-trained large models for natural language processing tasks;

[0094] The construction unit is used to construct multiple LoRA adapters as MoE experts beside multiple linear layers of the large model;

[0095] The designation unit is used to establish an expert routing S in the j-layer t module of the large model. jt Specify different LoRA adapters for input token data x;

[0096] The fine-tuning unit is used to freeze the large model cause weights and update the LoRA adapter parameters according to the low-rank matrix size r in the selected LoRA adapter and the parameters to be fine-tuned, so as to obtain a fine-tuned large model for the natural language processing task.

[0097] This embodiment also proposes an electronic device, including a memory and a processor; a computer program is stored on the memory; when the computer program is executed on the processor, the above-mentioned LoRA-MoE-based large model fine-tuning method is implemented.

[0098] This embodiment also proposes a computer-readable storage medium, on which computer instructions are stored; when the computer instructions are executed on the above-mentioned electronic device, the above-mentioned LoRA-MoE-based large model fine-tuning method is implemented.

[0099] The other parts of this embodiment are the same as any one of the above-mentioned embodiments 1-2, so they will not be repeated here.

[0100] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any simple modification or equivalent change made to the above embodiment based on the technical essence of the present invention shall fall within the protection scope of the present invention.

Claims

1. A large model fine-tuning method based on LoRA-MoE, characterized in that: The specific steps include: Step S1: Obtain the dataset and pre-trained large model for the natural language processing task; Step S2: Build multiple LoRA adapters as MoE experts next to multiple linear layers of the large model; Step S3: In the j-layer t module of the large model, establish an expert routing S jt Specify different LoRA adapters for input token data x; Step S4: According to the low-rank matrix size r in the selected LoRA adapter and the parameters to be fine-tuned, freeze the reason weights of the large model and update the LoRA adapter parameters to obtain the fine-tuned large model for the natural language processing task.

2. A large model fine-tuning method based on LoRA-MoE according to claim 1, characterized in that: The specific operation of step S2 is: in the weight matrix of the j-th layer t module of the large model Next to N j For the low-rank matrix A jt , low-rank matrix B jt As N j N adapters j MoE experts.

3. A large model fine-tuning method based on LoRA-MoE according to claim 2, characterized in that: The step S2 further includes: adjusting the number of LoRA adapters according to different layers of the large model, so that the number of experts in the upper layer is greater than the number of experts in the lower layer.

4. A large model fine-tuning method based on LoRA-MoE according to claim 2, characterized in that: The low-rank matrix A jt Initialized according to random Gaussian distribution, the low rank matrix B jt Initialized to 0, satisfying Ensure that the rank of the low-rank matrix is ​​much smaller than the rank of the weight matrix r<<min(d q ,d p ).

5. The large model fine-tuning method based on LoRA-MoE according to claim 1 is characterized in that: The specific operation of step S3 is to establish a trainable expert weight matrix W r jt , calculate the gate score F of MOE expert according to the set TOPK parameters and the input oken data x i jt .

6. A large model fine-tuning method based on LoRA-MoE according to claim 4, characterized in that: The step S4 specifically includes the following steps: Step S41: Based on the calculated gating score F i jt Construct the loss function L lbc , and applied to the load balancing loss of each layer of the large model; Step S42: Based on the input token data x and the large model weight matrix The product of the expert routing S jt From N j The top K MoE experts with the highest gate control scores are selected by the MoE experts, and the result h of the j-layer t module of the large model is obtained. jt That is, K pairs of low-rank matrices The product h of the input token data x and its routing weight jt ; Step S43: According to the set hyperparameters, product h jt , loss function L lbc , and get the final loss function L total .

7. A large model fine-tuning system based on LoRA-MoE, used to execute the large model fine-tuning method based on LoRA-MoE according to claim 1; characterized in that: It includes acquisition unit, construction unit, specification unit and fine-tuning unit; The acquisition unit is used to obtain data sets and pre-trained large models for natural language processing tasks; The construction unit is used to construct multiple LoRA adapters as MoE experts next to multiple linear layers of the large model; The designated unit is used to establish an expert routing S in the j-layer t module of the large model. jt Specify different LoRA adapters for input token data x; The fine-tuning unit is used to freeze the large model cause weights and update the LoRA adapter parameters according to the low-rank matrix size r in the selected LoRA adapter and the parameters to be fine-tuned, so as to obtain a fine-tuned large model for the natural language processing task.

8. An electronic device, characterized in that: It includes a memory and a processor; a computer program is stored on the memory; when the computer program is executed on the processor, the large model fine-tuning method based on LoRA-MoE according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions; when the computer instructions are executed on the electronic device according to claim 8, the large model fine-tuning method based on LoRA-MoE according to any one of claims 1 to 6 is implemented.

Images