Large language model parameter fine tuning method, system and device, storage medium and product

Abstract

The invention discloses a large language model parameter fine tuning method, system and device, a storage medium and a product, and relates to the field of natural language processing. The fine tuning method comprises the steps that a pre-training model and a parameter matrix of the pre-training model are acquired, a parameter fine tuning module is arranged on a self-attention layer of the pre-training model, the parameter fine tuning module comprises LoRA modules of different task types and a router, each LoRA module comprises a general feature matrix and a specific feature matrix, and the general feature matrixes of the multiple LoRA modules are shared; obtaining a fine tuning data set; loading and freezing a parameter matrix of the pre-training model, and initializing parameters of a parameter fine tuning module; and performing fine tuning on the parameters of the parameter fine tuning module based on the fine tuning data set to obtain a fine-tuned large language model. Parameters of the MoE architecture are remarkably reduced, it is ensured that the model captures differences of various tasks to the maximum extent, and the cross-task generalization of the model is ensured.

Description

Technical Field

[0001] The present application relates to the field of natural language processing technology, and in particular to a large language model parameter fine-tuning method, system, device, storage medium and product. Background Art

[0002] The emergence of Large Language Models (LLMs) has greatly promoted the development of Natural Language Processing (NLP) technology and laid a solid foundation for technologies with wide applicability. However, as the parameter scale increases, the full-parameter tuning (FP-tuning) process requires a lot of computing and memory resources. In order to strike a balance between resource requirements and effectiveness, the research community has increasingly turned to Efficient Parameter Fine-tuning (PEFT) methods, among which LoRA (Low-Rank Adaptation) has become the most common and effective choice. However, due to the inherent differences and diversity of various tasks, training LLMs through LoRA with multifaceted functions faces significant challenges. Figure 1 The scores of different datasets [OBQA (i.e. OpenBookQA), MCTest, Arc-e and Arc-c] in FP-tuning or LoRA are shown by Figure 1 It can be seen that while FP-tuning performs competitively in the multi-task mixed training data setting, ordinary LoRA shows a decline. This decline highlights the challenges posed by the heterogeneity and imbalance of training data, which leads to interference between data from different tasks, thereby reducing the performance of ordinary LoRA on in-domain tasks. In essence, ordinary LoRA is very sensitive to the configuration of training data.

[0003] As we all know, Mixture-of-Experts (MoE) networks have shown significant advantages in fusing multiple functions. In particular, the integration of MoE and LoRA is a promising approach to utilize MoE in a parameter-efficient way. This approach significantly reduces the training cost while retaining domain knowledge by introducing a limited number of domain-specific parameters. Currently, there are several works dedicated to applying MoE to LoRA. Some methods directly use the trained LoRA linear combination; some methods apply the combination of MoE and LoRA to different backbone models; and some methods train a LoRA module for each different task type and use a routing mechanism to integrate the LoRA module under a shared LLM. However, these methods are not enough to solve the problem of data conflict between different domains during LoRA training. There are three main challenges: (1) The MoE architecture emphasizes the unique attributes of each LoRA and ignores the general knowledge transfer between different LoRAs, which hinders cross-task generalization in LLM; (2) A large number of trained LoRA modules are required (one LoRA module for each task); (3) Multiple LoRAs will increase the number of parameters and computational cost. Summary of the invention

[0004] In response to the above-mentioned deficiencies, the present application provides a large language model parameter fine-tuning method, system, device, storage medium and product. On the basis of the classic MoE architecture, all experts in each layer share a common feature matrix while retaining the specific feature matrix of each expert. The sharing operation significantly reduces the parameters of the MoE architecture; applying the dropout strategy to the common feature matrix helps to balance the updates between the common feature matrix and the specific feature matrix.

[0005] In a first aspect, the present application provides a large language model parameter fine-tuning method, comprising: Obtain a pre-trained model and its parameter matrix, and set a parameter fine-tuning module in the self-attention layer of the pre-trained model; The parameter fine-tuning module includes a plurality of LoRA modules of different task types and a router, each of the LoRA modules includes a general feature matrix and a specific feature matrix, and the general feature matrices of the plurality of LoRA modules are shared; the router is used to select a suitable LoRA module for the target task; Get a fine-tuning dataset; Loading and freezing the parameter matrix of the pre-trained model, and initializing the parameters of the parameter fine-tuning module; The parameters of the parameter fine-tuning module are fine-tuned based on the fine-tuning data set to obtain a fine-tuned large language model.

[0006] Further, initializing the parameters of the parameter fine-tuning module includes: The universal feature matrix of each LoRA module is initialized using a random Gaussian distribution, and the specific feature matrix of each LoRA module is initialized to zero.

[0007] Furthermore, the router selects the first K LoRA modules for the target task by calculating the gating score; wherein the calculation formula of the gating score is: ; in, Indicates In the parameter fine-tuning module of the layer The gating score of each LoRA module; Indicates The input features of the self-attention layer of the layer; Represents the kth of the first K LoRA modules; Indicates The weight matrix of the router in the layer parameter fine-tuning module; represents the probability distribution function.

[0008] Furthermore, in the fine-tuning stage, the dropout strategy is used to update the universal feature matrix of each LoRA module; wherein the update formula of the universal feature matrix is: ; ; ; in, represents the updated general feature matrix; represents the general feature matrix before updating; represents a binary mask matrix, which is a mask probability matrix From Bernoulli drawn and generated from a distribution; represents dot product; represents the intermediate feature matrix.

[0009] Furthermore, in the fine-tuning stage, the calculation formula of the output feature of the self-attention layer is: ; in, Indicates The output features of the self-attention layer of the layer; Indicates The input features of the self-attention layer of the layer; Represents the parameter matrix of the pre-trained model; and Both represent scaling parameters; K Indicates the number of LoRA modules selected by the router; Indicates In the parameter fine-tuning module of the layer The gating score of each LoRA module; Indicates In the parameter fine-tuning module of the layer The common feature matrix of LoRA modules; Indicates In the parameter fine-tuning module of the layer Specific feature matrix of each LoRA module.

[0010] Furthermore, in the fine-tuning stage, the load balancing loss is added when calculating the loss value of the pre-trained model; wherein the specific calculation formula of the load balancing loss is: ; ; in, Indicates a loss of load balancing; Indicates The gating score of each LoRA module; K Indicates the number of LoRA modules selected by the router; Indicates that the The number of tokens for each LoRA module; represents all input features; Indicates The input features of the self-attention layer of the layer.

[0011] In a second aspect, the present application provides a large language model parameter fine-tuning system, comprising: A first acquisition module is used to acquire a pre-trained model and its parameter matrix, and set a parameter fine-tuning module in the self-attention layer of the pre-trained model; wherein the parameter fine-tuning module includes a plurality of LoRA modules of different task types and a router, each of the LoRA modules includes a general feature matrix and a specific feature matrix, and the general feature matrices of the plurality of LoRA modules are shared; the router is used to select a suitable LoRA module for the target task; The second acquisition module is used to acquire a fine-tuning dataset; A loading module, used to load and freeze the parameter matrix of the pre-trained model and initialize the parameters of the parameter fine-tuning module; A fine-tuning module is used to fine-tune the parameters of the parameter fine-tuning module based on the fine-tuning data set to obtain a fine-tuned large language model.

[0012] In a third aspect, the present application also provides an electronic device, including a memory, a processor, and a computer program / instructions stored on the memory, wherein the processor executes the computer program / instructions to implement the steps in the large language model parameter fine-tuning method as described above in the present application.

[0013] In a fourth aspect, the present application also provides a computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the steps in the large language model parameter fine-tuning method as described above in the present application.

[0014] In a fifth aspect, the present application also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps in the large language model parameter fine-tuning method described above in the present application.

[0015] The beneficial effects of this application are: This application is based on the classic MoE architecture and sets a parameter fine-tuning module for the self-attention layer. The parameter fine-tuning module includes multiple LoRA modules (i.e., experts). All LoRA modules in each self-attention layer share a common feature matrix while retaining the specific feature matrix of each LoRA module, which significantly reduces the parameters of the MoE architecture. The common feature matrix and the specific feature matrix of each LoRA module are tightly coupled, ensuring that the model captures the differences of various tasks to the maximum extent and ensures the generalization of the model across tasks.

[0016] This application adopts the dropout strategy to update the universal feature matrix of each LoRA module, balancing the update between the universal feature matrix and the specific feature matrix, and avoiding the problem of overfitting in the LoRA module. This application not only promotes a more balanced information exchange between different LoRA modules, but also alleviates the problems related to parameter redundancy and optimization imbalance. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only one embodiment of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 A schematic diagram of the scores of different data sets in FP-tuning or LoRA in the background technology of this application; Figure 2 This is a flow chart of the large language model parameter fine-tuning method in an embodiment of the present application; Figure 3 This is a diagram of the large language model architecture in the embodiment of this application; Figure 4 This is a schematic diagram of a parameter fine-tuning module in an embodiment of the present application; Figure 5 This is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0019] The following is a clear and complete description of the technical solutions in this application in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0020] The technical solution of the present application is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0021] Example 1 Figure 2 FIG. 2 shows a flow chart of a method for fine-tuning large language model parameters provided by the present application. Figure 2 As shown, in step S1, a pre-trained model and its parameter matrix are obtained, and a parameter fine-tuning module (i.e., MoSLD) is set in the self-attention layer of the pre-trained model.

[0022] In the area of ​​efficient parameter fine-tuning, LoRA introduces the concept of training only two low-rank matrices as an alternative to dense layer updates. In other words, LoRA reformulates the parameter fine-tuning process in LLM as a low-rank decomposition. Specifically, Eq. captures this decomposition, where represents the parameter matrix of LLM, Represents the parameter matrix The dimension of A matrix representing the updates during the fine-tuning phase. and is low-rank and trainable.

[0023] In order to achieve the migration of common features between different tasks and capture shared common knowledge, this application designs a sharing mechanism based on the basic principles of LoRA. Figure 3 As shown in Figure 2, for a large language model with L layers of Transformers, in the Layer Allocation experts and created A low-rank matrix pair , each expert is a LoRA module, each LoRA module corresponds to a task type and includes a general feature matrix and a specific feature matrix . General feature matrix In the The layer is shared between all LoRA modules, i.e. , retaining the specific feature matrix of each LoRA module Through this sharing mechanism, only Layer MoE architecture retention A universal feature matrix significantly reduces the parameters of the MoE architecture while forcing the model to maximize the capture of shared universal knowledge across various tasks. Although most parameters in each LoRA module are shared, due to the tight coupling between universal features and specific features, differences can still be learned in the specific feature matrix of each LoRA module, ensuring the generalization of the model across tasks.

[0024] This application uses a routing mechanism The LoRA modules are integrated under the large language model. Specifically, Figure 3 and Figure 4 As shown, for the pre-trained model (i.e., LLM), layer, The LoRA modules are simultaneously incorporated next to the self-attention layer. Before the layer, a router is inserted to guide the selection of different LoRA modules. That is, the router is used to select the appropriate LoRA module for the target task. The router of this embodiment uses probability distribution to select the most matching expert for a given input feature. Similar to the traditional MoE, the router of this application selects the front end according to the gated score. K The calculation formula of the gated score is: (1) in, Indicates In the parameter fine-tuning module of the layer The gating score of each LoRA module; Indicates The input features of the self-attention layer of the layer; Represents the kth of the first K LoRA modules; Indicates The weight matrix of the router in the layer parameter fine-tuning module; represents the probability distribution function.

[0025] Next, go to step S2. At step S2, obtain a fine-tuning dataset.

[0026] Next, go to step S3. In step S3, load and freeze the parameter matrix of the pre-trained model, and initialize the parameters of the parameter fine-tuning module.

[0027] For the pre-trained model Layer, whose parameter fine-tuning module includes The common feature matrix of each LoRA module is initialized by random Gaussian distribution, and the specific feature matrix of each LoRA module is initialized to zero.

[0028] Next, go to step S4. In step S4, the parameters of the parameter fine-tuning module are fine-tuned based on the fine-tuning dataset to obtain a fine-tuned large language model.

[0029] Considering that the general feature matrix is ​​updated more frequently than the specific feature matrix during the fine-tuning stage, and overfitting is prone to occur in the LoRA module, this application adopts the dropout strategy to update the general feature matrix of each LoRA module. The dropout strategy randomly ignores a certain proportion of parameter updates during each fine-tuning. This strategy helps prevent over-reliance on specific parameters and promotes robust learning by introducing noise, thereby improving the generalization ability of the model. Specifically, the mask probability From Bernoulli Extract from the distribution and generate a binary mask matrix , each element of the general feature matrix is ​​expressed in terms of probability Independent value 1 (retain) or with probability The value is 0 (discarded). The update formula of the general feature matrix is: (2) (3) (4) in, represents the updated general feature matrix; represents the general feature matrix before updating; represents a binary mask matrix; represents dot product; Represents the intermediate feature matrix.

[0030] In the fine-tuning phase, for the first Layer, input features Multiply it with the parameter matrix of the self-attention layer, and then perform weighted summation with the hybrid shared multi-LoRA to obtain the output features of the self-attention layer. The specific formula is: (5) in, Indicates The output features of the self-attention layer of the layer; Indicates The input features of the self-attention layer of the layer; Represents the parameter matrix of the pre-trained model (i.e. ), , Represent the weight matrices of the query vector and value vector of the self-attention layer, Figure 3 In The weight matrix representing the key vector of the self-attention layer; and Both represent scaling parameters; K Indicates the number of LoRA modules selected by the router; Indicates In the parameter fine-tuning module of the layer The gating score of each LoRA module; Indicates In the parameter fine-tuning module of the layer The common feature matrix of LoRA modules; Indicates In the parameter fine-tuning module of the layer Specific feature matrix of each LoRA module.

[0031] In order to achieve uniform distribution of LoRA modules in the fine-tuning stage, load balancing loss is added to the conventional loss of the pre-trained model. In this embodiment, the specific calculation formula of load balancing loss is: (6) (7) in, Indicates a loss of load balancing; Indicates The gating score of each LoRA module; K Indicates the number of LoRA modules selected by the router; Indicates that the The number of tokens for each LoRA module; represents all input features; Indicates The input features of the self-attention layer of the layer.

[0032] like Figure 3 As shown in Figure 2, the output features of the self-attention layer are summed and normalized by the Transformer and the feed-forward layer to obtain In a large language model with L layers of Transformers, steps S1 to S4 are repeated L times to obtain the final output.

[0033] In order to evaluate the effectiveness of the parameter fine-tuning module (i.e., MoSLD) of this application, experiments were conducted on 6 common sense reasoning datasets, including common sense question answering tasks (OBQA, CommonSense QA (CSQA)), reading comprehension tasks (Race, MCTest), and subject knowledge question answering tasks (Arc-e and Arc-c).

[0034] Table 1 shows the experimental results of various baselines in single and mixed settings in different datasets. As can be seen from Table 1, LoRA significantly outperforms other baselines, improving by 2.33% and 27.87% over FP-tuning (single) and Prefix-tuning (single), respectively; MoLoRA, SiRA, MoLA, and MixLoRA lag behind LoRA by 1.20%, 1.06%, 1.98%, and 1.00%, respectively, which shows that simply combining LoRA and MoE does not bring advantages in a single domain dataset. After establishing a robust baseline in a single setting, it can be seen from Table 1 that the performance of LoRA deteriorates, lagging behind FP-tuning (70.38%) by 1.23 percentage points; on the contrary, applying the MoE framework to LoRA, i.e., MoLoRA, SiRA, MoLA, and MixLoRA, can obtain scores of 69.61%, 69.35%, 70.00%, and 69.95%, respectively, indicating that MoE is suitable for multi-task scenarios, while MoLA is the best performing baseline in mixed settings. Further comparison between the single and hybrid settings shows that FP-tuning and MoLA in the hybrid setting improve by 3.14% and 2.41% respectively compared to the single setting. However, the performance of LoRA in the hybrid setting decreases by 0.42% compared to the single setting, indicating a conflict between the multi-task data and the adverse effects of the hybrid strategy on performance.

[0035] Table 1 Results of different methods on the in-domain test set of 6 common sense reasoning datasets

[0036] Note: The results in Table 1 are the average of three random runs (p < 0.01 under t-test). In Table 1, the full name of MoLORA is Mixture of Low-Rank Adaptation, and the corresponding paper is: Ted Zadouri et al, "Pushing mixture of experts to the limit: Extremely parameter efficient moe for instruction tuning".arXiv:2309.05444, 2023.9.11; the full name of SiRA is Sparse mixture of low rank adaptation; the corresponding paper of MoLA is: "Higher Layers Need More LoRA Experts" (2024) GitHub: github.com / GCYZSL / MoLA, MixLoRA means conditional LoRA mixture, and the corresponding paper is: D.Li et al., "MixLoRA: Enhancing large language models fine-tuning with LoRA-based Mixture of Experts," CoRR, vol. abs / 2404.15159, 2024; MoSL represents the parameter adjustment module of this application without the Dropout strategy; MoSLD represents the parameter adjustment module of this application.

[0037] Example 2 The large language model parameter fine-tuning system provided in the embodiment of the present application includes a first acquisition module, a second acquisition module, a loading module and a fine-tuning module.

[0038] The first acquisition module is used to obtain a pre-trained model and its parameter matrix, and set a parameter fine-tuning module in the self-attention layer of the pre-trained model; wherein the parameter fine-tuning module includes a plurality of LoRA modules of different task types and a router, each LoRA module includes a general feature matrix and a specific feature matrix, and the general feature matrices of multiple LoRA modules are shared; the router is used to select a suitable LoRA module for the target task; The second acquisition module is used to acquire a fine-tuning dataset; The loading module is used to load and freeze the parameter matrix of the pre-trained model and initialize the parameters of the parameter fine-tuning module; The fine-tuning module is used to fine-tune the parameters of the parameter fine-tuning module based on the fine-tuning dataset to obtain a fine-tuned large language model.

[0039] In some embodiments, the large language model parameter fine-tuning system may be combined with features of the large language model parameter fine-tuning method in Embodiment 1 of the present application, and vice versa, which will not be elaborated herein.

[0040] Example 3 The embodiment of the present invention further provides an electronic device, such as Figure 5 As shown, the electronic device includes: a memory, a processor, and a computer program / instruction stored in the memory, and the processor executes the computer program / instruction to implement the large language model parameter fine-tuning method in Example 1 of the present application.

[0041] Although not shown, the electronic device includes a processor that can perform various appropriate operations and processes according to the programs and / or data stored in the read-only memory (ROM) or the programs and / or data loaded from the storage part into the random access memory (RAM). The processor can be a multi-core processor or can include multiple processors. In some embodiments, the processor can include a general-purpose main processor and one or more special coprocessors, such as a central processing unit, a graphics processing unit (GPU), a neural network processor (NPU), a digital signal processor (DSP), etc. In RAM, various programs and data required for device operation are also stored. The processor, ROM, and RAM are connected to each other via a bus. The input / output (I / O) interface is also connected to the bus.

[0042] The processor and memory are used together to execute the program / instructions stored in the memory. When the program / instructions are executed by the computer, the methods, steps or functions described in the above embodiments can be implemented.

[0043] Although not shown, an embodiment of the present invention further provides a computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the large language model parameter fine-tuning method in Embodiment 1 of the present application.

[0044] Readable storage media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0045] Although not shown, an embodiment of the present invention further provides a computer program product, including: a computer program / instruction, which, when executed by a processor, implements the large language model parameter fine-tuning method in Embodiment 1 of the present application.

[0046] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they are aware of the basic creative concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application. Obviously, those skilled in the art may make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for fine-tuning parameters of a large language model, characterized in that: The fine-tuning method comprises: Obtain a pre-trained model and its parameter matrix, and set a parameter fine-tuning module in the self-attention layer of the pre-trained model; The parameter fine-tuning module includes a plurality of LoRA modules of different task types and a router, each of the LoRA modules includes a general feature matrix and a specific feature matrix, and the general feature matrices of the plurality of LoRA modules are shared; the router is used to select a suitable LoRA module for the target task; Get a fine-tuning dataset; Loading and freezing the parameter matrix of the pre-trained model, and initializing the parameters of the parameter fine-tuning module; The parameters of the parameter fine-tuning module are fine-tuned based on the fine-tuning data set to obtain a fine-tuned large language model.

2. The large language model parameter fine-tuning method according to claim 1, characterized in that: Initializing the parameters of the parameter fine-tuning module includes: The universal feature matrix of each LoRA module is initialized using a random Gaussian distribution, and the specific feature matrix of each LoRA module is initialized to zero.

3. The large language model parameter fine-tuning method according to claim 1, characterized in that: The router selects the first K LoRA modules for the target task by calculating the gating score; wherein the calculation formula of the gating score is: ； in, Indicates In the parameter fine-tuning module of the layer The gating score of each LoRA module; Indicates The input features of the self-attention layer of the layer; represents the kth of the first K LoRA modules; Indicates The weight matrix of the router in the layer parameter fine-tuning module; represents the probability distribution function.

4. The large language model parameter fine-tuning method according to claim 1, characterized in that: In the fine-tuning stage, the dropout strategy is used to update the universal feature matrix of each LoRA module; wherein the update formula of the universal feature matrix is: ； ； ； in, represents the updated general feature matrix; represents the general feature matrix before updating; represents a binary mask matrix, which is a mask probability matrix From Bernoulli drawn and generated from a distribution; represents dot product; represents the intermediate feature matrix.

5. The large language model parameter fine-tuning method according to claim 1, characterized in that: In the fine-tuning stage, the calculation formula of the output feature of the self-attention layer is: ； in, Indicates The output features of the self-attention layer of the layer; Indicates The input features of the self-attention layer of the layer; Represents the parameter matrix of the pre-trained model; and Both represent scaling parameters; K Indicates the number of LoRA modules selected by the router; Indicates In the parameter fine-tuning module of the layer The gating score of each LoRA module; Indicates In the parameter fine-tuning module of the layer The common feature matrix of LoRA modules; Indicates In the parameter fine-tuning module of the layer Specific feature matrix of each LoRA module.

6. The large language model parameter fine-tuning method according to claim 1, characterized in that: In the fine-tuning stage, the load balancing loss is added when calculating the loss value of the pre-trained model; wherein the specific calculation formula of the load balancing loss is: ； ； in, Indicates a loss of load balancing; Indicates The gating score of each LoRA module; K Indicates the number of LoRA modules selected by the router; Indicates that the The number of tokens for each LoRA module; represents all input features; Indicates The input features of the self-attention layer of the layer.

7. A large language model parameter fine-tuning system, characterized in that: The fine-tuning system comprises: A first acquisition module is used to acquire a pre-trained model and its parameter matrix, and set a parameter fine-tuning module in the self-attention layer of the pre-trained model; wherein the parameter fine-tuning module includes a plurality of LoRA modules of different task types and a router, each of the LoRA modules includes a general feature matrix and a specific feature matrix, and the general feature matrices of the plurality of LoRA modules are shared; the router is used to select a suitable LoRA module for the target task; The second acquisition module is used to acquire a fine-tuning dataset; A loading module, used to load and freeze the parameter matrix of the pre-trained model and initialize the parameters of the parameter fine-tuning module; A fine-tuning module is used to fine-tune the parameters of the parameter fine-tuning module based on the fine-tuning data set to obtain a fine-tuned large language model.

8. An electronic device comprising a memory, a processor, and a computer program / instruction stored in the memory, characterized in that: The processor executes the computer program / instructions to implement the steps in the large language model parameter fine-tuning method as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instruction is executed by a processor, the steps in the large language model parameter fine-tuning method as described in any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the steps in the large language model parameter fine-tuning method as described in any one of claims 1 to 6 are implemented.

Images