Model fine-tuning method and apparatus, and electronic device and non-volatile storage medium

By optimizing the routing mechanism based on prompt information and the load balancing loss function, the storage and computing overhead issues of LoRA in a multi-tenant environment are resolved, achieving efficient model fine-tuning and inference speed.

WO2026237282A1PCT designated stage Publication Date: 2026-11-19CHINA TELECOM ARTIFICIAL INTELLIGENCE TECHNOLOGY (BEIJING) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2025/132402
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2025-05-16
Filing Date
2025-11-04
Publication Date
2026-11-19

AI Technical Summary

Technical Problem

In multi-tenant environments, existing low-rank adaptation methods (LoRA) suffer from significant increases in storage and computational overhead, leading to increased inference latency.

Method used

By using a routing mechanism based on prompts, the most suitable low-rank adapter (LoRA adaptation layer) is pre-selected and the selection is reused during inference, avoiding recalculation of the selection strategy when each lexical is generated. The load-balanced loss function and learnable activation function are used to optimize the model parameters.

Benefits of technology

It reduces computational overhead during inference, improves the efficiency and adaptability of the model in multi-tenant environments, and solves the technical problems of high storage and computational overhead.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2025132402_19112026_PF_FP_ABST
    Figure CN2025132402_19112026_PF_FP_ABST
Patent Text Reader

Abstract

Disclosed in the present application are a model fine-tuning method and apparatus, and an electronic device and a non-volatile storage medium. The method comprises: acquiring prompt information, wherein the prompt information is used for representing the type of a task that needs to be executed by a model; inputting the prompt information into a backbone network of the model, and selecting a low-rank adapter corresponding to the prompt information before the prompt information passes through the backbone network for the first time and the model generates the first token, wherein the low-rank adapter is used for performing fine-tuning training on the model with respect to the task type corresponding to the prompt information, and different low-rank adapters correspond to different task types; and during the process of the model generating subsequent tokens, reusing the selected low-rank adapter to perform fine-tuning training on the model.
Need to check novelty before this filing date? Find Prior Art

Description

Model fine-tuning methods, devices, electronic equipment, and non-volatile storage media

[0001] Related applications

[0002] This application claims priority to Chinese patent application filed on May 16, 2025, application number 2025106394132, entitled “Model fine-tuning method, apparatus, electronic device and non-volatile storage medium”, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of artificial intelligence technology, and more specifically, to a model fine-tuning method, apparatus, electronic device, and non-volatile storage medium. Background Technology

[0004] In recent years, large-scale pre-trained language models (LLMs) have made significant progress in a wide range of natural language processing (NLP) tasks. However, in practical applications, fine-tuning is still necessary to adapt LLMs to specific tasks. Although full fine-tuning can achieve good performance, its computational cost and storage requirements are extremely high due to the massive size of LLMs (typically reaching billions or even hundreds of billions of parameters). Therefore, parameter-efficient fine-tuning (PEFT) methods have gradually become mainstream. These methods adapt to specific tasks by adjusting only a small number of parameters, thereby significantly reducing computational resource consumption.

[0005] Currently, Low-Rank Adaptation (LoRA), as one of the most successful PEFT methods, is widely used in the fine-tuning of LLMs. LoRA learns task-specific adjustments by adding a low-rank trainable matrix to the frozen pre-trained model weights without modifying the original parameters. This method significantly reduces storage overhead, allowing the same LLM to be efficiently adapted to multiple tasks through different LoRA adaptation layers. Although LoRA performs well on many tasks, it still faces some challenges in multi-tenant environments. In such environments, multiple LoRA adaptation layers need to be stored simultaneously and switched according to different tasks, causing memory overhead to increase linearly with the number of tasks and leading to increased inference latency.

[0006] There is currently no effective solution to the above problems. Summary of the Invention

[0007] According to one aspect of the embodiments of this application, a model fine-tuning method is provided, comprising: acquiring cue information, wherein the cue information is used to characterize the type of task that the model needs to perform; inputting the cue information into the backbone network of the model, and before the cue information passes through the backbone network for the first time and before the model generates the first lexical unit, selecting a low-rank adapter corresponding to the cue information, wherein the low-rank adapter is used to fine-tune the model for the task type corresponding to the cue information, and different low-rank adapters correspond to different task types; and reusing the selected low-rank adapter to fine-tune the model during the generation of subsequent lexical units by the model.

[0008] Optionally, selecting the low-rank adapter corresponding to the prompt information includes: performing an embedding transformation on the prompt information to obtain the embedding representation corresponding to the prompt information; inputting the embedding representation corresponding to the prompt information into the backbone network of the model to obtain the hidden state information corresponding to the prompt information; using a pooling layer to perform a pooling operation on the hidden state information to obtain the target vector corresponding to the prompt information, wherein the pooling operation includes at least one of the following: pooling based on the last word, mean pooling, max pooling, and attention-based pooling; and using a low-rank adaptation router to determine the low-rank adapter corresponding to the prompt information based on the target vector.

[0009] Optionally, the low-rank adapter router includes: a linear layer, a normalization function, and a selection function. Using the low-rank adapter router, determining the low-rank adapter corresponding to the prompt information based on the target vector includes: using the linear layer to calculate the original probability of the target vector being assigned to each low-rank adapter; using the normalization function to normalize the original probabilities to obtain the probability distribution corresponding to the low-rank adapter, where the probability distribution represents the target probability of each low-rank adapter being selected, and the target probability represents the degree of matching between the low-rank adapter and the task type represented by the prompt information; using the selection function, based on the probability distribution, selecting a preset number of low-rank adapters with the highest target probability from all low-rank adapters, and activating the selected low-rank adapters.

[0010] Optionally, the backbone network of the model includes: a Transformer backbone network; reusing the selected low-rank adapters to fine-tune the model includes: during the generation of each subsequent word except the first word, using a low-rank adapter router to determine the activated low-rank adapters at the Transformer level, and using the activated low-rank adapters to fine-tune the model when each word is generated.

[0011] Optionally, the method further includes: during model training, determining the cue percentage for each low-rank adapter in each training batch, wherein each training batch's training dataset contains multiple training samples, and each training sample contains: cue information and output labels corresponding to the cue information; the cue percentage is used to characterize the proportion of cue information allocated to low-rank adapters in the current training batch; determining the average probability quality index for each low-rank adapter in each training batch, wherein the average probability quality index is the average probability of a low-rank adapter being selected for each training sample in the current training batch; determining the load balancing loss function value based on the cue percentage and the average probability quality index, wherein the load balancing loss function value is used to characterize the degree of equitable use of each low-rank adapter during training; and adjusting the model parameters based on the load balancing loss function value, wherein the model parameters include: parameters of the low-rank adapter and the low-rank adapter router.

[0012] Optionally, the method further includes: during model training, obtaining the word sequence output by the model based on the prompt information; and determining the model output loss function value based on the word sequence and the output labels in the training dataset corresponding to the prompt information, wherein the model output loss function value is used to characterize the degree of error between the model's output value and the true output labels in the training dataset.

[0013] Optionally, the method further includes: adjusting the model parameters based on the model output loss function value, and adjusting the learnable parameters in the activation function corresponding to the low-rank adaptive router.

[0014] According to another aspect of the embodiments of this application, a model fine-tuning apparatus is also provided, comprising: an information acquisition module for acquiring prompt information, wherein the prompt information is used to characterize the type of task that the model needs to perform; an expert selection module for inputting the prompt information into the backbone network of the model, and selecting a low-rank adapter corresponding to the prompt information before the prompt information first passes through the backbone network and before the model generates the first lexical unit, wherein the low-rank adapter is used to fine-tune the model for the task type corresponding to the prompt information, and different low-rank adapters correspond to different task types; and a fine-tuning training module for reusing the selected low-rank adapter to fine-tune the model during the subsequent lexical unit generation process of the model.

[0015] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a memory and a processor, the processor being configured to run a program stored in the memory, wherein the program executes a model fine-tuning method during runtime.

[0016] According to another aspect of the embodiments of this application, a non-volatile storage medium is also provided, the non-volatile storage medium including a stored computer program, wherein the device where the non-volatile storage medium is located executes a model fine-tuning method by running the computer program.

[0017] According to another aspect of the embodiments of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the steps of a model fine-tuning method.

[0018] Details of one or more embodiments of this application are set forth in the following drawings and description. Other features, objects, and advantages of this application will become apparent from the specification, drawings, and claims. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0020] Figure 1 is a hardware structure block diagram of a computer terminal (or electronic device) for implementing a model fine-tuning method according to an embodiment of this application;

[0021] Figure 2 is a schematic diagram of a model fine-tuning method provided according to an embodiment of this application;

[0022] Figure 3 is a schematic diagram of an algorithm architecture for LoRA routing fine-tuning for large models provided according to an embodiment of this application;

[0023] Figure 4 is a schematic diagram of a model fine-tuning device provided according to an embodiment of this application. Detailed Implementation

[0024] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0026] To facilitate a better understanding of the embodiments of this application by those skilled in the art, some technical terms or nouns involved in the embodiments of this application are explained as follows:

[0027] Large language models are a type of artificial intelligence model designed to understand and generate human language. They are trained on large amounts of text data and can perform a wide range of tasks, including text summarization, translation, sentiment analysis, and more.

[0028] MOE (Mixture of Experts) is a deep learning architecture whose core idea is to use multiple "expert" sub-models to process different types of data and select the most suitable expert to compute the output through a gating network. This improves the computational efficiency and performance of the model and is commonly used in large-scale NLP, computer vision, and recommender systems. MOE mainly consists of expert networks, gating networks, and weighted summation. During execution, only some experts are activated, thereby reducing computational costs.

[0029] LoRA (Low-Rank Adaptation) fine-tuning is a parameter-efficient fine-tuning (PEFT) method designed for large-scale pre-trained models. It learns task-specific adjustments by adding a low-rank matrix to the pre-trained weights without changing the original model parameters, thus significantly reducing the number of training parameters and memory usage. LoRA is suitable for tasks such as NLP, improving fine-tuning efficiency while maintaining model performance, and is particularly suitable for large model deployments and multi-task adaptation.

[0030] PEFT (Parameter-Efficient Fine-Tuning) is a method for optimizing large-scale pre-trained models, aiming to reduce the number of parameters required for fine-tuning, improve computational efficiency, and maintain model performance. Common techniques include LoRA (Low-Rank Adaptation), Adapter, Prefix-Tuning, and Prompt-Tuning. These methods achieve efficient fine-tuning by adjusting a small number of parameters or additional layers, rather than modifying the entire model weights. They are widely used in NLP and other tasks, and are particularly suitable for deploying large models and adapting to multiple tasks.

[0031] Related technologies such as MOE LoRA (Mixture of Experts LoRA) and DoRA (Dynamic Routing LoRA) employ a multi-expert system-style LoRA selection mechanism. This involves training multiple LoRA modules and using a gating network to select the appropriate LoRA adaptation layer for the current input during inference. While this approach reduces the storage overhead of single-task LoRA, a key issue remains: during inference, the LoRA selection strategy needs to be recalculated for each token generation, leading to significant computational overhead and reduced inference speed.

[0032] To address the aforementioned issues, this application provides a solution, proposing a more efficient LoRA variant—MiLoRA (Mixture of Low-Rank Adaptation). MiLoRA primarily optimizes the inference computation overhead of MOE LoRA and proposes a prompt-aware routing mechanism, which will be described in detail below.

[0033] According to an embodiment of this application, a method embodiment for model fine-tuning is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0034] The method embodiments provided in this application can be executed in a mobile terminal, computer terminal, or similar computing device. Figure 1 shows a hardware structure block diagram of a computer terminal (or electronic device) for implementing a model fine-tuning method. As shown in Figure 1, the computer terminal 10 (or electronic device) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that the structure shown in Figure 1 is only illustrative and does not limit the structure of the above-described electronic device. For example, the computer terminal 10 may also include more or fewer components than shown in Figure 1, or have a different configuration than shown in Figure 1.

[0035] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or electronic device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0036] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the model fine-tuning method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the above-mentioned model fine-tuning method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0037] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0038] The display can be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or electronic device).

[0039] Under the above operating environment, this application embodiment provides a model fine-tuning method. Figure 2 is a schematic diagram of the flow of a model fine-tuning method provided according to an embodiment of this application. As shown in Figure 2, the method includes the following steps:

[0040] Step S202: Obtain prompt information, wherein the prompt information is used to characterize the type of task that the model needs to perform;

[0041] Step S204: Input the prompt information into the backbone network of the model, and before the prompt information passes through the backbone network for the first time and before the model generates the first word, select the low-rank adapter corresponding to the prompt information. The low-rank adapter is used to fine-tune the model for the task type corresponding to the prompt information. Different low-rank adapters correspond to different task types.

[0042] Step S206: In the process of generating subsequent lexical units by the model, the selected low-rank adapter is reused to fine-tune the model.

[0043] Through the above steps, a routing mechanism based on prompt information is adopted. By introducing prompt information, the most suitable low-rank adapter (LoRA adaptation layer) is pre-determined during inference, avoiding the need to recalculate the selection strategy when each token is generated. This achieves the goal of improving inference speed and reducing computational overhead during inference, thereby solving the technical problem of large storage and computational overhead in multi-tenant environments for fine-tuning methods in related technologies.

[0044] Figure 3 is a schematic diagram of an algorithm architecture for LoRA routing with large model fine-tuning according to an embodiment of this application. The model fine-tuning method in steps S202 to S206 of the embodiment of this application will be further described below with reference to Figure 3.

[0045] In this embodiment, expert routes can be selected based on user-input prompts. Specifically, in this embodiment, low-rank adapters (LoRAs) can be selected as experts, and a suitable LoRA module is chosen according to the routing mechanism. The overall process includes the following steps: inputting prompts, calculating the hidden state h, selecting a suitable expert module through the prompt routing mechanism, obtaining the top K experts, applying the LoRA adapter (reusing expert selection), generating a token, and continuing decoding until the sequence ends. The process is described in detail below.

[0046] First, receive the prompt input by the user and calculate the hidden state based on the prompt. The specific steps are as follows.

[0047] In some embodiments of this application, selecting the low-rank adapter corresponding to the prompt information includes the following steps: performing an embedding transformation on the prompt information to obtain the embedding representation corresponding to the prompt information; inputting the embedding representation corresponding to the prompt information into the backbone network of the model to obtain the hidden state information corresponding to the prompt information; using a pooling layer to perform a pooling operation on the hidden state information to obtain the target vector corresponding to the prompt information, wherein the pooling operation includes at least one of the following: pooling based on the last word, mean pooling, max pooling, and attention-based pooling; and using a low-rank adapter router to determine the low-rank adapter corresponding to the prompt information based on the target vector.

[0048] Specifically, the input prompt is converted into an embedding that the model can understand, and then the embedded prompt is input into the backbone network of the model (taking the Transformer model as an example) to compute the hidden state.

[0049] In this embodiment, the hidden state of prompt This refers to the hidden representation obtained before the Lth layer of the Transformer model. Specifically, when the input prompt obtains its hidden representation through the backbone network of the Transformer model, the hidden state of the Lth layer is denoted as... Where n p 'd' represents the length of the prompt, and 'd' represents the hidden state at each layer. The computational details differ depending on the model. Taking the BERT model's embedding as an example, the embedding of each token is calculated as follows: token embedding, position embedding, and paragraph embedding. Corresponding to the above

[0050] Therefore, after embedding and before being input into the first transformer layer, the dimension of the prompt is...

[0051] After obtaining the hidden state information, the expert selection strategy can be determined. Specifically, a pooling layer can be used to aggregate the hidden state information. That is, h l =Pooler(H l ).

[0052] In this embodiment, the Pooler operation may include, but is not limited to: 1) Pooling based on the last token: using the representation of the last token in the prompt message as h l ; 2) Mean pooling 3) Max pooling 4) Attention-based pooling.

[0053] Then, a low-rank adapter router (LoRA router) can be used. l For the target vector h l The process involves determining the low-rank adapter corresponding to the prompt message, that is, assigning the currently input prompt message to the most suitable LoRA expert (low-rank adapter). The specific steps are as follows.

[0054] In some embodiments of this application, the low-rank adapter router includes: a linear layer, a normalization function, and a selection function. Using the low-rank adapter router, determining the low-rank adapter corresponding to the prompt information based on the target vector includes the following steps: using the linear layer, calculating the original probability that the target vector is assigned to each low-rank adapter; using the normalization function, normalizing the original probability to obtain the probability distribution corresponding to the low-rank adapter, wherein the probability distribution is used to characterize the target probability of each low-rank adapter being selected, and the target probability is used to characterize the degree of matching between the low-rank adapter and the task type represented by the prompt information; using the selection function, based on the probability distribution, selecting a preset number of low-rank adapters with the highest target probability from all low-rank adapters, and activating the selected low-rank adapters.

[0055] Specifically, the target vector h l Through activation function g and LoRA router R l Processing, R l It is responsible for assigning the current input suggestion to the most suitable LoRA expert. This router consists of: a linear layer, a normalization function (Softmax function), and a selection function (Top-k selection function). The linear layer is used to calculate h. lThe original probabilities are assigned to each LoRA expert (low-rank adapter). The Softmax function is then used to transform the output of the linear layer into a probability distribution, i.e., constructing the probability distribution of the LoRA experts. It ensures that the sum of the probabilities of all LoRA experts is 1, thus forming an efficient probability distribution. The output of the Softmax function represents the probability of each LoRA expert being selected; these probabilities reflect each expert's adaptability to the current input cue. Finally, the Top-k selection function is used to select the k experts with the highest probability quality from the probability distribution. The formal expression is shown in the following equation:

[0056] in, This is the router's weight matrix. During inference, the LoRA router dynamically selects the k most suitable experts and calls them only once before generating a new token. The activated LoRA experts are reused throughout the generation process.

[0057] In this embodiment, the activation decision of the LoRA router is calculated only when the input prompt information first passes through the Transformer backbone network, and the activated expert is determined before the generation of the first new token. Subsequently, this decision is reused throughout the generation process. During reuse, the activated LoRA expert is determined at the Transformer layer level, i.e., the Transformer module is selected to be modified by the corresponding LoRA module. The expert reuse strategy in this application is described below.

[0058] In some embodiments of this application, the backbone network of the model includes: a Transformer backbone network; reusing the selected low-rank adapter to fine-tune the model includes the following steps: during the generation of each subsequent word except the first word, a low-rank adapter router is used to determine the activated low-rank adapter at the Transformer level, and the activated low-rank adapter is used to fine-tune the model when each word is generated.

[0059] Specifically, in this embodiment, the activation decision of the LoRA router is calculated only once, when the input prompt first passes through the Transformer backbone network and before the generation of the first new token. These decisions are reused in subsequent generation steps. That is, the MiLoRA method only calculates which LoRA experts should be activated once, when the input prompt first passes through the Transformer network and before the generation of the first new token. These decisions are reused in subsequent generation steps and do not need to be recalculated in each generation step. Therefore, in each subsequent token generation process, the LoRA router determines the activated LoRA experts at the Transformer layer and selects which Transformer module is modified by its corresponding LoRA module. Through the LoRA router, the most suitable k experts can be dynamically selected for inference for each input prompt, instead of calling multiple routers in each generation step. Therefore, during training, the generation of each subsequent token will activate the corresponding Transformer layer LoRA experts for training, while inactive LoRA modules will not participate in model training.

[0060] In addition, in this application example, a load balancing loss term can be added to the training loss to improve the generalization ability and adaptability of the model, as detailed below.

[0061] In some embodiments of this application, the method further includes the following steps: during the training process of the model, determining the cue proportion corresponding to each low-rank adapter in each training batch, wherein the training dataset of each training batch contains multiple training samples, and each training sample contains: cue information and output labels corresponding to the cue information, the cue proportion is used to characterize the proportion of cue information allocated to low-rank adapters in the current training batch; determining the average probability quality index corresponding to each low-rank adapter in each training batch, wherein the average probability quality index is the average probability of low-rank adapters being selected for each training sample in the current training batch; determining the load balancing loss function value based on the cue proportion and the average probability quality index, wherein the load balancing loss function value is used to characterize the degree of balance in the use of each low-rank adapter during the training process; adjusting the model parameters of the model based on the load balancing loss function value, wherein the model parameters include: parameters of the low-rank adapter and the low-rank adapter router.

[0062] In some embodiments of this application, the method further includes the following steps: during model training, obtaining the word sequence output by the model based on the prompt information; determining the model output loss function value based on the word sequence and the output labels in the training dataset corresponding to the prompt information, wherein the model output loss function value is used to characterize the degree of error between the model's output value and the true output labels in the training dataset.

[0063] Specifically, the training process of the model includes the following steps: First, prepare the training dataset, in which the training samples include the input prompt information and the corresponding output label; divide the dataset into multiple batches, each batch containing a certain number of samples; during the forward propagation phase of training, input prompt information from the training batches is fed into the backbone network of the model, the hidden state of the prompt information is calculated, the LoRA router is used to select the most suitable LoRA expert based on the hidden state, and the selected LoRA expert is used to generate the output sequence.

[0064] When calculating the loss, on one hand, the task loss (i.e., the model output loss function value mentioned above) is calculated based on the output sequence generated by the model and the true output labels in the training dataset. On the other hand, the load balancing loss function value can also be calculated. Finally, the task loss and the load balancing loss are added together to obtain the total loss. Backpropagation is then performed on the total loss to calculate the gradient of the model parameters. The optimizer is then used to update the model parameters based on the gradient.

[0065] The specific method for calculating the load balancing loss function value is as follows.

[0066] Assume that training batch B contains N B A sample, defined This represents the percentage of hints allocated to the i-th LoRA expert at level l:

[0067] in, Let be the probability of the j-th expert output from the l-th layer of the router. Let the average probability quality index received by the i-th expert be:

[0068] The load balancing loss function value is:

[0069] By introducing a load balancing loss, the model training process encourages a more balanced use of all LoRA experts, thereby improving the model's generalization and adaptability. This loss function is particularly suitable for multi-task learning scenarios, where the model needs to switch and adapt between different tasks.

[0070] On the other hand, in the embodiments of this application, the model can also automatically learn the optimal activation function configuration during training, thereby improving the model's performance and adaptability. Specifically, as follows.

[0071] In some embodiments of this application, the method further includes the following steps: adjusting the model parameters of the model based on the model output loss function value, and adjusting the learnable parameters in the activation function corresponding to the low-rank adaptive router.

[0072] Specifically, in related technologies, the activation function in the PEFT module is usually set to the ReLU function, and the PEFT modules of different Transformer layers typically use the same activation function. However, the embodiments of this application make the activation function learnable during training, as shown in the following formula:

[0073] Among them, a j and b i These are learnable parameters. The Padé function can approximate common activation functions and adaptively adjusts during training. This design avoids the hassle of manually tuning hyperparameters and can automatically learn the optimal activation function for different tasks.

[0074] When fine-tuning downstream tasks, rational activation functions can be used to learn the activation functions of the LoRA router. Let the set of parameters for the learnable activation function be θ, and the other parameter set for the LoRA router and LoRA expert be Ω. In this embodiment, θ is treated as an architectural parameter and optimized together with Ω through bi-level optimization.

[0075] Taking into account both the model output loss function and the load balancing loss function, adjusting the model parameters is a key step in achieving efficient fine-tuning in this technical solution. By simultaneously optimizing these two loss functions, the model can not only improve its output quality on specific tasks but also ensure the balanced use of all low-rank adapters, avoiding resource waste and performance bottlenecks.

[0076] The expert selection mechanism in this application only selects experts during the Prompt calculation (i.e., the LoRA router is only called before the input prompt and the generation of the first token), and the same expert selection is reused for subsequent token generation, reducing the computational overhead during inference. Compared to the route calculation method where every token is calculated, the computational complexity is reduced from O(N) to O(k). Furthermore, although there are still multiple experts at each layer in this application, only a fixed number of experts participate in the calculation for each generated sequence. This is due to the top-k expert selection strategy, so the number of experts selected at each layer in this application is also less than that of traditional solutions in related technologies. In addition, this application uses a learnable activation function to replace the original ReLU activation function, and further improves the activation function and other parameters of the LoRA module through a two-layer optimization approach.

[0077] According to an embodiment of this application, an embodiment of a model fine-tuning device is also provided. Figure 4 is a schematic diagram of the structure of a model fine-tuning device provided according to an embodiment of this application. As shown in Figure 4, the device includes:

[0078] The information acquisition module 40 is used to acquire prompt information, wherein the prompt information is used to characterize the type of task that the model needs to perform;

[0079] The expert selection module 42 is used to input the prompt information into the backbone network of the model and select the low-rank adapter corresponding to the prompt information before the prompt information passes through the backbone network for the first time and before the model generates the first word. The low-rank adapter is used to fine-tune the model for the task type corresponding to the prompt information. Different low-rank adapters correspond to different task types.

[0080] The fine-tuning training module 44 is used to reuse the selected low-rank adapter to fine-tune the model during the subsequent generation of lexical units by the model.

[0081] Optionally, selecting the low-rank adapter corresponding to the prompt information includes: performing an embedding transformation on the prompt information to obtain the embedding representation corresponding to the prompt information; inputting the embedding representation corresponding to the prompt information into the backbone network of the model to obtain the hidden state information corresponding to the prompt information; using a pooling layer to perform a pooling operation on the hidden state information to obtain the target vector corresponding to the prompt information, wherein the pooling operation includes at least one of the following: pooling based on the last word, mean pooling, max pooling, and attention-based pooling; and using a low-rank adaptation router to determine the low-rank adapter corresponding to the prompt information based on the target vector.

[0082] Optionally, the low-rank adapter router includes: a linear layer, a normalization function, and a selection function. Using the low-rank adapter router, determining the low-rank adapter corresponding to the prompt information based on the target vector includes: using the linear layer to calculate the original probability of the target vector being assigned to each low-rank adapter; using the normalization function to normalize the original probabilities to obtain the probability distribution corresponding to the low-rank adapter, where the probability distribution represents the target probability of each low-rank adapter being selected, and the target probability represents the degree of matching between the low-rank adapter and the task type represented by the prompt information; using the selection function, based on the probability distribution, selecting a preset number of low-rank adapters with the highest target probability from all low-rank adapters, and activating the selected low-rank adapters.

[0083] Optionally, the backbone network of the model includes: a Transformer backbone network; reusing the selected low-rank adapters to fine-tune the model includes: during the generation of each subsequent word except the first word, using a low-rank adapter router to determine the activated low-rank adapters at the Transformer level, and using the activated low-rank adapters to fine-tune the model when each word is generated.

[0084] Optionally, the model fine-tuning device is further configured to: determine the cue percentage corresponding to each low-rank adapter in each training batch during model training, wherein each training batch's training dataset contains multiple training samples, and each training sample contains: cue information and output labels corresponding to the cue information; the cue percentage is used to characterize the proportion of cue information allocated to low-rank adapters in the current training batch; determine the average probability quality index corresponding to each low-rank adapter in each training batch, wherein the average probability quality index is the average probability of a low-rank adapter being selected for each training sample in the current training batch; determine the load balancing loss function value based on the cue percentage and the average probability quality index, wherein the load balancing loss function value is used to characterize the degree of equitable use of each low-rank adapter during training; and adjust the model parameters based on the load balancing loss function value, wherein the model parameters include: parameters of the low-rank adapter and the low-rank adapter router.

[0085] Optionally, the model fine-tuning device is also used to: during model training, acquire the word sequence output by the model based on the prompt information; and determine the model output loss function value based on the word sequence and the output labels in the training dataset that correspond to the prompt information, wherein the model output loss function value is used to characterize the degree of error between the model's output value and the true output labels in the training dataset.

[0086] Optionally, the model fine-tuning device is also used to: adjust the model parameters of the model based on the model output loss function value, and adjust the learnable parameters in the activation function corresponding to the low-rank adaptive router.

[0087] It should be noted that each module in the above model fine-tuning device can be a program module (e.g., a set of program instructions to implement a certain function) or a hardware module. For the latter, it can be manifested in the following forms, but is not limited to them: each of the above modules is manifested as a processor, or the functions of each of the above modules are implemented by a processor.

[0088] It should be noted that the model fine-tuning device provided in this embodiment can be used to execute the model fine-tuning method shown in Figure 2. Therefore, the relevant explanations and descriptions of the above model fine-tuning method also apply to the embodiments of this application, and will not be repeated here.

[0089] This application embodiment also provides a non-volatile storage medium, which includes a stored computer program. The device containing the non-volatile storage medium executes the following model fine-tuning method by running the computer program: acquiring prompt information, wherein the prompt information is used to characterize the type of task the model needs to perform; inputting the prompt information into the backbone network of the model, and before the prompt information first passes through the backbone network and before the model generates its first lexical unit, selecting a low-rank adapter corresponding to the prompt information, wherein the low-rank adapter is used to fine-tune the model for the task type corresponding to the prompt information, and different low-rank adapters correspond to different task types; during the subsequent lexical unit generation process of the model, reusing the selected low-rank adapter to fine-tune the model.

[0090] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the model fine-tuning method described in various embodiments of this application: obtaining prompt information, wherein the prompt information is used to characterize the type of task that the model needs to perform; inputting the prompt information into the backbone network of the model, and before the prompt information passes through the backbone network for the first time and before the model generates the first lexical unit, selecting a low-rank adapter corresponding to the prompt information, wherein the low-rank adapter is used to fine-tune the model for the task type corresponding to the prompt information, and different low-rank adapters correspond to different task types; and reusing the selected low-rank adapter to fine-tune the model during the subsequent lexical unit generation process.

[0091] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0092] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0093] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0094] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0095] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0096] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0097] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A model fine-tuning method, comprising: Obtain prompt information, wherein the prompt information is used to characterize the type of task that the model needs to perform; The prompt information is input into the backbone network of the model, and before the prompt information passes through the backbone network for the first time and before the model generates the first word, a low-rank adapter corresponding to the prompt information is selected. The low-rank adapter is used to fine-tune the model for the task type corresponding to the prompt information, and different low-rank adapters correspond to different task types. During the subsequent generation of lexical units by the model, the selected low-rank adapter is reused to fine-tune the model.

2. The model fine-tuning method according to claim 1, wherein selecting the low-rank adapter corresponding to the prompt information includes: The prompt information is subjected to embedding transformation to obtain the embedded representation corresponding to the prompt information; The embedded representation corresponding to the prompt information is input into the backbone network of the model to obtain the hidden state information corresponding to the prompt information; A pooling layer is used to perform pooling operations on the hidden state information to obtain the target vector corresponding to the prompt information. The pooling operation includes at least one of the following: pooling based on the last word, mean pooling, max pooling, and attention-based pooling. A low-rank adapter router is used to determine the low-rank adapter corresponding to the prompt information based on the target vector.

3. The model fine-tuning method of claim 2, wherein the low-rank adaptation router comprises: a linear layer, a normalization function, a selection function. Using a low-rank adapter router, the low-rank adapter corresponding to the prompt information is determined based on the target vector, including: Using the linear layer, the original probability of the target vector being assigned to each low-rank adapter is calculated; The original probability is normalized using the normalization function to obtain the probability distribution corresponding to the low-rank adapter. The probability distribution is used to characterize the target probability of each low-rank adapter being selected, and the target probability is used to characterize the degree of matching between the low-rank adapter and the task type represented by the prompt information. Using the selection function and based on the probability distribution, a preset number of low-rank adapters with the highest target probability are selected from all low-rank adapters, and the selected low-rank adapters are activated.

4. The model fine-tuning method of claim 3, wherein the backbone network of the model comprises: Transformer backbone network; Reusing the selected low-rank adapter to fine-tune the model includes: During the generation of each subsequent word in the model, except for the first word, the low-rank adapter router is used to determine the activated low-rank adapter at the Transformer level, and the model is fine-tuned using the activated low-rank adapter when each word is generated.

5. The model fine-tuning method according to claim 1, wherein the method further comprises: During the training process of the model, the cue proportion corresponding to each low-rank adapter in each training batch is determined. Each training batch contains multiple training samples in its training dataset, and each training sample contains: the cue information and the output label corresponding to the cue information. The cue proportion is used to characterize the proportion of the low-rank adapters assigned to the cue information in the current training batch. Determine the average probability quality index corresponding to each low-rank adapter in each training batch, wherein the average probability quality index is the average probability of the low-rank adapter being selected for each training sample in the current training batch; Based on the aforementioned prompt ratio and the aforementioned average probability quality index, a load balancing loss function value is determined, wherein the load balancing loss function value is used to characterize the degree of equitable use of each of the aforementioned low-rank adapters during the training process; Based on the load balancing loss function value, the model parameters of the model are adjusted, wherein the model parameters include the parameters of the rank adapter and the low-rank adapter router.

6. The model fine-tuning method according to claim 5, wherein the method further comprises: During model training, the word sequence output by the model based on the prompt information is obtained; Based on the word sequence and the output labels in the training dataset corresponding to the prompt information, the model output loss function value is determined, wherein the model output loss function value is used to characterize the degree of error between the model's output value and the true output labels in the training dataset.

7. The model fine-tuning method according to claim 6, wherein the method further comprises: Based on the loss function value output by the model, the model parameters are adjusted, as are the learnable parameters in the activation function corresponding to the low-rank adaptive router.

8. A model fine-tuning device, comprising: An information acquisition module is used to acquire prompt information, wherein the prompt information is used to characterize the type of task that the model needs to perform; An expert selection module is used to input the prompt information into the backbone network of the model, and select the low-rank adapter corresponding to the prompt information before the prompt information passes through the backbone network for the first time and before the model generates the first word. The low-rank adapter is used to fine-tune the model for the task type corresponding to the prompt information, and different low-rank adapters correspond to different task types. The fine-tuning training module is used to reuse the selected low-rank adapter to fine-tune the model during the subsequent generation of lexical units by the model.

9. An electronic device comprising: A memory and a processor, the processor being configured to run a program stored in the memory, wherein the program, when running, executes the model fine-tuning method according to any one of claims 1 to 7.

10. A non-transitory storage medium comprising a stored computer program, wherein, The device containing the non-volatile storage medium executes the model fine-tuning method according to any one of claims 1 to 7 by running the computer program.

11. A computer program product comprising a computer program, wherein the computer program, when executed by a processor, implements the steps of the model fine-tuning method of any one of claims 1 to 7.