Hybrid expert model training method, system, device and medium based on lora technology

Abstract

The application discloses a mixed expert model training method and system based on LORA technology, equipment and medium, mainly related to the technical field of expert model training, to solve the problem that in the multi-task learning scene, a single LoRA adjustment method may not be able to fully cope with the diversity and subtle differences between different tasks, and it is difficult to take into account the needs of all tasks, thereby leading to the decline of overall performance. Including: in the process of training by using the preset fine-tuning data set, the LoRA layer is dynamically selected for parameter adjustment from the several LoRA layers by the gating network according to the training data corresponding to the current task, the output of the LoRA layer is adjusted by the gating network, and the output of the pre-training network is obtained; after completing the training of the multi-task pre-training model and the LoRA layer, the parameters of the multi-task pre-training model and the LoRA layer are frozen.

Description

Technical Field

[0001] The present application relates to the technical field of hybrid expert models, and in particular to a hybrid expert model training method, system, device and medium based on LORA technology. Background Art

[0002] In the field of deep learning, particularly natural language processing, pre-trained models, trained on large amounts of text data, have demonstrated remarkable capabilities in handling complex language tasks. However, directly applying these pre-trained models to specific tasks or datasets often fails to achieve optimal results, necessitating fine-tuning to adapt to new requirements.

[0003] Existing full-parameter fine-tuning methods primarily use LoRA (Long Range Radio) technology to insert trainable low-rank matrices into specific layers of a pre-trained model to perform parameter updates. However, in multi-task learning (MTL) scenarios, a single LoRA tuning approach may not adequately address the diversity and nuances of different tasks. Because different tasks may require the model to focus on different features or knowledge representations, a single LoRA tuning approach often fails to address the needs of all tasks, resulting in a decrease in overall performance.

[0004] Therefore, in order to further improve the processing capability of the model in multi-task learning scenarios, a hybrid expert model training method, system, equipment and medium based on LoRa technology are urgently needed to solve the problem that in multi-task learning scenarios, a single LoRA adjustment method may not be able to fully cope with the diversity and subtle differences between different tasks, and it is difficult to take into account the needs of all tasks, thus leading to a decline in overall performance. Summary of the Invention

[0005] In response to the above-mentioned shortcomings of the existing technology, the present application provides a hybrid expert model training method, system, device and medium based on LoRa technology to solve the problem that in the existing multi-task learning scenario, a single LoRA adjustment method may not be able to fully cope with the diversity and subtle differences between different tasks, and it is difficult to take into account the needs of all tasks, thus leading to a decline in overall performance.

[0006] In the first aspect, the present application provides a hybrid expert model training method based on LORA technology, the method comprising:

[0007] According to a preset performance selection procedure, a multi-task pre-trained model that meets the preset performance requirements is determined from several multi-task pre-trained models; a preset fine-tuning dataset of the Chinese environment corresponding to the tasks in the multi-task pre-trained model is created; and the preset fine-tuning dataset is used to train several LoRA layers corresponding to the Transformer block in the multi-task pre-trained model; wherein, in the process of training using the preset fine-tuning dataset, a LoRA layer is dynamically selected from several LoRA layers according to the training data corresponding to the current task through a gating network for parameter adjustment, and the output of the LoRA layer is adjusted through the gating network to obtain the output of the pre-trained network; after completing the training of the multi-task pre-trained model and the LoRA layer, the parameters of the multi-task pre-trained model and the LoRA layer are frozen; the maximum number of tokens processed by each LoRA layer in a single batch and the preset multi-objective optimization strategy are obtained to perform load balancing optimization for the operation of the multi-task pre-trained model.

[0008] The hybrid expert model training method provided in the embodiment of the present application solves the problem that in the existing multi-task learning scenario, a single LoRA adjustment method may not be able to fully cope with the diversity and subtle differences between different tasks, and it is difficult to take into account the needs of all tasks, thus resulting in a decline in overall performance, by adding data of several LoRA layers to the Transformer block in the training multi-task pre-training model. In addition, the LoRA layer is dynamically selected from several LoRA layers according to the training data corresponding to the current task through the gating network to adjust the parameters, and the output of the LoRA layer is adjusted through the gating network to obtain the output of the pre-training network; the above enables the present application to intelligently select and combine different LoRA layers for fine-tuning according to the characteristics of different tasks, so as to achieve accurate adaptation and efficient training between tasks. At the same time, on the basis of retaining the stable inheritance of LoRA expert characteristics, the LoRA combination can be optimized to improve the overall performance, and the contributions between different LoRA experts can be balanced to avoid the risk of overfitting.

[0009] In one implementation of the present application, the method further includes: in the model inference stage of the trained multi-task pre-training model, following the parameter freezing principle, freezing the parameters of the multi-task pre-training model and the LoRA layer except for the gated network.

[0010] In one implementation of the present application, a gated network dynamically selects a LoRA layer from several LoRA layers for parameter adjustment based on the training data corresponding to the current task, specifically including:

[0011] By formula:

[0012] ;

[0013] ;

[0014] = ;

[0015] Calculate the activation weight of the i-th LoRA layer corresponding to task t ;in, represents the embedding representation of task t; 、 、 and is a preset value; N represents the total number of LoRA layers;

[0016] By formula:

[0017] , in order to adapt to task t, adjust and update the parameters of the Lth layer of the multi-task pre-training model ;

[0018] in, Indicates the parameters of the Lth layer before updating, Represents the low-rank update matrix provided by the i-th LoRA layer to the L-th layer.

[0019] In one implementation of the present application, the output of the LoRA layer is adjusted by a gating network to obtain the output of the pre-trained network, specifically including:

[0020] Get the input data x of the LoRA layer;

[0021] By formula:

[0022] , splice and normalize the output of all LoRA layers to obtain data ;

[0023] in, represents the output of the i-th LoRA layer,

[0024] and ,in, represents the feedforward neural network layer, LN represents layer normalization, Represents the parameters learned by the i-th LoRA layer of the current task during training, and N represents the number of all LoRA layers;

[0025] By formula:

[0026] , calculate the gated input of each LoRA layer;

[0027] By formula:

[0028] , calculate the weight of the i-th LoRA layer; where τ represents the preset temperature parameter;

[0029] By formula:

[0030] , calculate the weighted sum of all LoRA layers output by the gating network;

[0031] By formula:

[0032] , the output of the gating network Added to the output of multi-task pre-trained models In the example above, we get the output of the pre-trained network .

[0033] In one implementation of the present application, the method further includes:

[0034] By formula:

[0035] ;

[0036] ;

[0037] Calculate the total loss function of the multi-task pre-trained model ;

[0038] in, represents the loss function corresponding to the task, represents a preset constant, and ∈(0,1), represents the true label of the configuration, Represents the preset balancing hyperparameters.

[0039] In one implementation of the present application, the maximum number of tokens processed by each LoRA layer in a single batch and the preset multi-objective optimization strategy are obtained to perform load balancing optimization of the multi-task pre-training model operation, specifically including:

[0040] The formula for setting the expert capacity threshold is:

[0041] , calculate the maximum number of tokens processed by the i-th LoRA layer in a single batch ;

[0042] in, represents the minimum value of the preset capacity threshold of the i-th LoRA layer, represents the maximum value of the preset capacity threshold of the i-th LoRA layer, represents the average of the first k-1 batch requests, Indicates the preset adjustment step size, Indicates the maximum number of tokens processed by the i-th LoRA layer in a single batch before optimization.

[0043] In one implementation of the present application, the maximum number of tokens processed by each LoRA layer in a single batch and the preset multi-objective optimization strategy are obtained to perform load balancing optimization of the multi-task pre-training model operation, specifically including:

[0044] By solving the preset multi-objective optimization strategy formula:

[0045] ;

[0046] Calculate and obtain the solution set X that reaches the maximum number of iterations or the improvement of the solution set is less than the preset threshold;

[0047] in, Indicates the variance of the LoRA layer load corresponding to the current solution set X, represents the average system response time corresponding to the current solution set X, Indicates the resource utilization corresponding to the current solution set X, 、 、 Indicates the preset weight coefficient.

[0048] In the second aspect, the present application provides a hybrid expert model training system based on LORA technology, the system comprising:

[0049] The pre-trained model preparation module is used to select programs according to preset performance and determine a multi-task pre-trained model that meets the preset performance requirements from several multi-task pre-trained models; the dataset construction module is used to create a preset fine-tuning dataset for the Chinese environment corresponding to the tasks in the multi-task pre-trained model; the network fine-tuning module is used to use the preset fine-tuning dataset to train several LoRA layers corresponding to the Transformer block in the multi-task pre-trained model; wherein, in the process of training using the preset fine-tuning dataset, the LoRA layer is dynamically selected from several LoRA layers according to the training data corresponding to the current task through the gating network for parameter adjustment, and the output of the LoRA layer is adjusted through the gating network to obtain the output of the pre-trained network; the model processing module is used to freeze the parameters of the multi-task pre-trained model and the LoRA layer after completing the training of the multi-task pre-trained model and the LoRA layer; obtain the maximum number of tokens processed by each LoRA layer in a single batch and the preset multi-objective optimization strategy, and perform load balancing optimization for the operation of the multi-task pre-trained model.

[0050] In a third aspect, the present application provides a hybrid expert model training device based on LORA technology, the device comprising:

[0051] processor;

[0052] and a memory having executable code stored thereon, which, when the executable code is executed, causes the processor to execute any of the above-mentioned hybrid expert model training methods based on LORA technology.

[0053] In a fourth aspect, the present application provides a non-volatile computer storage medium on which computer instructions are stored. When the computer instructions are executed, they implement a hybrid expert model training method based on LORA technology as described above.

[0054] Those skilled in the art will appreciate that this application has at least the following beneficial effects:

[0055] The present application discloses a hybrid expert model training method, system and medium based on LoRa technology. By configuring several LoRa layers in the preset layers or modules of the multi-task pre-training model, introducing technical means such as multi-task LoRA fine-tuning based on the gating mechanism, hierarchical LoRA combination optimization, parameter freezing strategy, and intelligent load balancing, it achieves accurate adaptation and efficient training for different tasks, providing a new solution for multi-task learning in the field of deep learning, and solving the problem that in the existing multi-task learning scenario, a single LoRA adjustment method may not be able to fully cope with the diversity and subtle differences between different tasks, and it is difficult to take into account the needs of all tasks, resulting in a decrease in overall performance. In addition, the gated network dynamically selects the LoRA layer from several LoRA layers for parameter adjustment based on the training data corresponding to the current task, and adjusts the output of the LoRA layer through the gated network to obtain the output of the pre-trained network; the above enables the present application to intelligently select and combine different LoRA layers for fine-tuning according to the characteristics of different tasks to achieve accurate adaptation and efficient training between tasks. At the same time, it can optimize the LoRA combination and improve the overall performance on the basis of retaining the stable inheritance of LoRA expert characteristics, and balance the contributions between different LoRA experts to avoid the risk of overfitting. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Some embodiments of the present disclosure are described below with reference to the accompanying drawings, in which:

[0057] Figure 1 This is a flow chart of a hybrid expert model training method based on LORA technology provided in an embodiment of the present application.

[0058] Figure 2 This is a schematic diagram of the internal structure of a hybrid expert model training system based on LORA technology provided in an embodiment of the present application.

[0059] Figure 3 This is a schematic diagram of the internal structure of a hybrid expert model training device based on LORA technology provided in an embodiment of the present application. DETAILED DESCRIPTION

[0060] It is to be understood that the embodiments described hereinbelow are merely preferred embodiments of the present disclosure and do not represent the only possible implementation of the present disclosure. The preferred embodiments are merely used to explain the technical principles of the present disclosure and not intended to limit the scope of protection of the present disclosure. Based on the preferred embodiments provided herein, any other embodiments obtained by those of ordinary skill in the art without creative efforts should fall within the scope of protection of the present disclosure.

[0061] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or apparatus that comprises a list of elements does not only include those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or apparatus. Without more limitations, the element defined by the phrase "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0062] The technical solutions of the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0063] The embodiments provide a hybrid expert model training method based on LORA technology, as shown in Figure 1 The method provided by the embodiments of the present application mainly includes the following steps:

[0064] Step 110, according to a preset performance selection program, determining a multi-task pre-training model meeting a preset performance requirement from a plurality of multi-task pre-training models.

[0065] It should be noted that the preset performance selection program is used to select a multi-task pre-training model with excellent performance (for example, Qwen2), and then the hardware resources are planned and configured. While ensuring the efficient and stable operation of the model, considering the parameter efficiency of LoRA, the selected model should have sufficient flexibility to adapt to different fine-tuning tasks.

[0066] Step 120, creating a preset fine-tuning data set of a Chinese environment corresponding to a task of the multi-task pre-training model.

[0067] It should be noted that the fine-tuning data set (preset fine-tuning data set) for the Chinese environment is constructed, covering scenarios such as abstract generation, intelligent question answering, and text classification. The data set should be diverse to comprehensively optimize the multi-task processing capability of the model. In the fine-tuning process of the model, the model parameters can be updated using the following formula:

[0068] ;

[0069] in, represents the fine-tuned model parameters, represents the pre-trained model parameters, represents the parameters learned by fine-tuning.

[0070] For multi-task learning scenarios, this application can define a composite loss function, which is the weighted sum of the loss functions of each task:

[0071] ;

[0072] in, is the total loss function, N is the number of tasks, is the loss function of the i-th task (such as cross entropy loss, mean square error loss, etc.), is the preset weight of the i-th task, which is used to balance the impact of different tasks on model training.

[0073] Step 130: Use a preset fine-tuning dataset to train several LoRA layers corresponding to the Transformer block in the multi-task pre-training model; wherein, during the training process using the preset fine-tuning dataset, a LoRA layer is dynamically selected from several LoRA layers according to the training data corresponding to the current task through a gating network for parameter adjustment, and the output of the LoRA layer is adjusted through the gating network to obtain the output of the pre-trained network.

[0074] It should be noted that in multi-task learning scenarios, different tasks may require the model to focus on different features or knowledge representations. As a parameter-efficient fine-tuning method, LoRA adapts to new tasks by adding low-rank matrices without adjusting all parameters of the pre-trained model. To further improve the performance of LoRA in multi-task scenarios, this application can treat each LoRA layer as an "expert" and design a gating network to intelligently select the most appropriate LoRA expert (LoRA layer) for fine-tuning.

[0075] In addition, the goal of the gating network is to assign a weight to each LoRA layer (i.e., expert) based on the characteristics or context information of the current task, thereby determining which LoRA layers (experts) should be activated for fine-tuning. For example, the gating network dynamically selects a LoRA layer from several LoRA layers for parameter adjustment based on the training data corresponding to the current task, specifically:

[0076] By formula:

[0077] ;

[0078] ;

[0079] = ;

[0080] Calculate the activation weight of the i-th LoRA layer corresponding to task t ;in, represents the embedding representation of task t; 、 、 and is a preset value; N represents the total number of LoRA layers;

[0081] By formula:

[0082] , in order to adapt to task t, adjust and update the parameters of the Lth layer of the multi-task pre-training model ;

[0083] in, Indicates the parameters of the Lth layer before updating, Represents the low-rank update matrix provided by the i-th LoRA layer to the L-th layer.

[0084] For further example, the algorithm pseudo code of this step can be:

[0085] Algorithm: Multi-task LoRA fine-tuning based on gating mechanism;

[0086] Input: pre-trained model, LoRA layer set { }, task set {T}, task embedding { };

[0087] Output: Fine-tuned model for each task;

[0088] 1. Initialize the gating network parameters;

[0089] 2. for each task t in T do;

[0090] 3. et = Get the embedding representation of task t;

[0091] 4. gt = softmax(MLP(et)) # Calculate the gating vector;

[0092] 5. for each layer L in pre-trained model do;

[0093] 6. # Update the layer parameters;

[0094] 7. end for use Fine-tune the model to suit task t;

[0095] 8. end for.

[0096] In addition, this application introduces a learnable gating function to achieve fine-grained control of weight distribution. Assume that the pre-trained model is defined by parameters θ, which contains multiple Transformer blocks. For each Transformer block, this application trains multiple LoRA parameter sets { }, where i = 0, 1, ..., N, where N is the number of LoRA layers. Given an input x ∈ R^{L×d}, where L and d represent the sequence length and input dimension respectively, the output of the LoRA layer is adjusted by the gating network to obtain the output of the pre-trained network. Specifically,

[0097] Get the input data x of the LoRA layer;

[0098] By formula:

[0099] , splice and normalize the output of all LoRA layers to obtain data ;

[0100] in, represents the output of the i-th LoRA layer,

[0101] and ,in, represents the feedforward neural network layer, LN represents layer normalization, Represents the parameters learned by the i-th LoRA layer of the current task during training, and N represents the number of all LoRA layers;

[0102] By formula:

[0103] , calculate the gated input of each LoRA layer;

[0104] By formula:

[0105] , calculate the weight of the i-th LoRA layer; where τ represents the preset temperature parameter, represents the gating function;

[0106] By formula:

[0107] , calculate the weighted sum of all LoRA layers output by the gating network;

[0108] By formula:

[0109] , the output of the gating network Added to the output of multi-task pre-trained models In the example above, we get the output of the pre-trained network .

[0110] In addition, in the hierarchical LoRA combination optimization, this application not only focuses on achieving fine-grained control of weights through gating functions, but also hopes to optimize this process so that the combined LoRA can achieve better performance on specific tasks. To this end, this application introduces Gating BaLancing Loss to balance the contributions between different LoRA experts and promote the overall performance of the model. This application uses entropy regularization to encourage a uniform distribution of weights:

[0111] ;

[0112] where δ is a small positive number, and ∈(0,1), which is used to avoid computational problems with the logarithmic function.

[0113] Combining standard task losses (such as cross entropy loss for classification tasks) and Gating BaLancing Loss, the overall training objective can be expressed as:

[0114] ;

[0115] Among them, y is the true label, is the task-dependent loss function, and λ is a hyperparameter that balances the two loss terms.

[0116] In this way, this application not only optimizes the LoRA combination to improve task performance, but also balances or prefers the contributions of different LoRA experts through GatingBaLancing Loss, thereby achieving more refined regulation.

[0117] Step 140: After completing the training of the multi-task pre-training model and the LoRA layer, freeze the parameters of the multi-task pre-training model and the LoRA layer.

[0118] In some embodiments, except for the gating function parameters, all trained LoRA parameters and pre-trained model basic parameters are frozen. This strategy helps retain expert knowledge and avoid overfitting. The formula is as follows:

[0119] ;

[0120] Here, θfrozen represents the frozen model parameters.

[0121] Step 150: Obtain the maximum number of tokens processed by each LoRA layer in a single batch and the preset multi-objective optimization strategy to perform load balancing optimization for the multi-task pre-training model operation.

[0122] It should be noted that this application can achieve dynamic resource allocation and efficient utilization by setting a reasonable expert capacity threshold to limit the maximum number of tokens each LoRA expert can process in a single batch. In addition to setting an expert capacity threshold to limit the maximum token processing capacity of each LoRA expert in a single batch, a more sophisticated algorithm can be introduced to dynamically adjust this threshold. A queue length prediction algorithm (based on the ARIMA model of time series) is used to estimate the request volume in the next few batches and adjust the expert capacity threshold accordingly.

[0123] As an example, to obtain the maximum number of tokens processed by each LoRA layer in a single batch, the specific number can be:

[0124] The formula for setting the expert capacity threshold is:

[0125] , calculate the maximum number of tokens processed by the i-th LoRA layer in a single batch ;

[0126] in, represents the minimum value of the preset capacity threshold of the i-th LoRA layer, represents the maximum value of the preset capacity threshold of the i-th LoRA layer, represents the average of the first k-1 batch requests, Indicates the preset adjustment step size, Indicates the maximum number of tokens processed by the i-th LoRA layer in a single batch before optimization.

[0127] In addition, the present application can also design a load balancing strategy based on multi-objective optimization, where the objective function can include not only the load balancing degree but also the system response time, resource utilization, etc. as optimization targets. The specific process can be:

[0128] By solving the preset multi-objective optimization strategy formula:

[0129] ;

[0130] Calculate and obtain the solution set X that reaches the maximum number of iterations or the improvement of the solution set is less than the preset threshold;

[0131] in, Indicates the variance of the LoRA layer load corresponding to the current solution set X, represents the average system response time corresponding to the current solution set X, Indicates the resource utilization corresponding to the current solution set X, 、 、 Indicates the preset weight coefficient.

[0132] In order to solve the above multi-objective optimization problem, the basic steps of the algorithm used in this application can be:

[0133] 1. Initialize a set of random solutions (called the population).

[0134] 2. Evaluate the objective function value for each solution.

[0135] 3. Generate a new solution set based on the objective function value selection, crossover and mutation operations.

[0136] 4. Repeat steps 2 and 3 until a stopping condition is met (e.g., the maximum number of iterations is reached or the improvement of the solution is less than a preset threshold).

[0137] The method also includes:

[0138] During the model inference phase, the parameter freezing principle is followed to ensure that all LoRA parameters and pre-trained model parameters, except for the gating function, remain unchanged. This helps improve the generalization ability and stability of the model under limited training data conditions. The relevant formula is as follows:

[0139] ;

[0140] Among them, θeffective is the set of parameters actually used for inference, which is determined by:

[0141] ;

[0142] here, e is the portion of the LoRA increment that is activated according to the gating function g(x) (or the empty set if all LoRA parameters are frozen). g(x) is a binary or continuous-valued function that decides whether to apply the LoRA increment (i.e., g(x)∈{0,1} or g(x)∈[0,1]) based on the input x and possible additional conditions.

[0143] In addition, this application Figure 2 A hybrid expert model training system based on LORA technology is provided in the embodiment of this application. Figure 2 As shown, the system provided in the embodiment of the present application mainly includes:

[0144] The pre-trained model preparation module 210 is used to select a multi-task pre-trained model that meets the preset performance requirements from a plurality of multi-task pre-trained models according to a preset performance selection procedure;

[0145] The data set construction module 220 is used to create a preset fine-tuning data set in the Chinese environment corresponding to the tasks in the multi-task pre-training model.

[0146] The network fine-tuning module 230 is used to train several LoRA layers corresponding to the Transformer block in the multi-task pre-training model using a preset fine-tuning data set; wherein, during the training process using the preset fine-tuning data set, the LoRA layer is dynamically selected from several LoRA layers according to the training data corresponding to the current task through the gating network for parameter adjustment, and the output of the LoRA layer is adjusted through the gating network to obtain the output of the pre-trained network.

[0147] The model processing module 240 is used to freeze the parameters of the multi-task pre-training model and the LoRA layer after completing the training of the multi-task pre-training model and the LoRA layer; obtain the maximum number of tokens processed by each LoRA layer in a single batch and the preset multi-objective optimization strategy, and perform load balancing optimization for the operation of the multi-task pre-training model.

[0148] The above is a method embodiment of the present application. Based on the same inventive concept, the present application embodiment also provides a hybrid expert model training device based on LORA technology. Figure 3 As shown, the device includes: a processor; and a memory on which executable code is stored. When the executable code is executed, the processor executes a hybrid expert model training method based on LORA technology as in the above embodiment.

[0149] Specifically, the server side determines a multi-task pre-trained model that meets the preset performance requirements from several multi-task pre-trained models according to a preset performance selection program; creates a preset fine-tuning data set for the Chinese environment corresponding to the tasks in the multi-task pre-trained model; uses the preset fine-tuning data set to train several LoRA layers corresponding to the Transformer block in the multi-task pre-trained model; wherein, in the process of training using the preset fine-tuning data set, the LoRA layer is dynamically selected from several LoRA layers according to the training data corresponding to the current task through the gating network for parameter adjustment, and the output of the LoRA layer is adjusted through the gating network to obtain the output of the pre-trained network; after completing the training of the multi-task pre-trained model and the LoRA layer, the parameters of the multi-task pre-trained model and the LoRA layer are frozen; the maximum number of tokens processed by each LoRA layer in a single batch and the preset multi-objective optimization strategy are obtained to perform load balancing optimization for the operation of the multi-task pre-trained model.

[0150] In addition, an embodiment of the present application also provides a non-volatile computer storage medium on which executable instructions are stored. When the executable instructions are executed, a hybrid expert model training method based on LORA technology as described above is implemented.

[0151] Thus far, the technical solutions of the present disclosure have been described in conjunction with the foregoing multiple embodiments. However, it is easy for those skilled in the art to understand that the scope of protection of the present disclosure is not limited to these specific embodiments. Without departing from the technical principles of the present disclosure, those skilled in the art may split and combine the technical solutions in the above-mentioned various embodiments, and may also make equivalent changes or replacements to the relevant technical features. Any changes, equivalent replacements, improvements, etc. made within the technical concepts and / or technical principles of the present disclosure will fall within the scope of protection of the present disclosure.

Claims

1. A hybrid expert model training method based on LORA technology, characterized in that: The method comprises: Determine, according to a preset performance selection procedure, a multi-task pre-trained model that meets preset performance requirements from a number of multi-task pre-trained models; Create a preset fine-tuning dataset for the Chinese environment corresponding to the tasks in the multi-task pre-training model; Using a preset fine-tuning dataset, train several LoRA layers corresponding to the Transformer block in the multi-task pre-trained model; wherein, during the training process using the preset fine-tuning dataset, a gating network dynamically selects a LoRA layer from several LoRA layers for parameter adjustment based on the training data corresponding to the current task, and adjusts the output of the LoRA layer through the gating network to obtain the output of the pre-trained network; After completing the training of the multi-task pre-trained model and the LoRA layer, freeze the parameters of the multi-task pre-trained model and the LoRA layer; Obtain the maximum number of tokens processed by each LoRA layer in a single batch and preset multi-objective optimization strategies to optimize the load balancing of multi-task pre-training model operations; specifically, The formula for setting the expert capacity threshold is: , calculate the maximum number of tokens processed by the i-th LoRA layer in a single batch ; in, represents the minimum value of the preset capacity threshold of the i-th LoRA layer, represents the maximum value of the preset capacity threshold of the i-th LoRA layer, represents the average of the first k-1 batch requests, Indicates the preset adjustment step size, Indicates the maximum number of tokens processed by the i-th LoRA layer in a single batch before optimization.

2. The hybrid expert model training method based on LORA technology according to claim 1, characterized in that: The method further comprises: In the model inference stage of the trained multi-task pre-trained model, follow the parameter freezing principle and freeze the parameters of the multi-task pre-trained model and LoRA layer except for the gating network.

3. The hybrid expert model training method based on LORA technology according to claim 1, characterized in that: The gated network dynamically selects a LoRA layer from several LoRA layers based on the training data corresponding to the current task for parameter adjustment, including: By formula: ； ； = ； Calculate the activation weight of the i-th LoRA layer corresponding to task t ;in, represents the embedding representation of task t; 、 、 and is a preset value; N represents the total number of LoRA layers; By formula: , in order to adapt to task t, adjust and update the parameters of the Lth layer of the multi-task pre-training model ; in, Indicates the parameters of the Lth layer before updating, Represents the low-rank update matrix provided by the i-th LoRA layer to the L-th layer.

4. The hybrid expert model training method based on LORA technology according to claim 1, characterized in that: The output of the LoRA layer is adjusted through the gating network to obtain the output of the pre-trained network, specifically including: Get the input data x of the LoRA layer; By formula: , splice and normalize the output of all LoRA layers to obtain data ; in, represents the output of the i-th LoRA layer, and ,in, represents the feedforward neural network layer, LN represents layer normalization, Represents the parameters learned by the i-th LoRA layer in the current task during training, and N represents the number of all LoRA layers; By formula: , calculate the gated input of each LoRA layer; By formula: , calculate the weight of the i-th LoRA layer; where τ represents the preset temperature parameter, represents the gating function; By formula: , calculate the weighted sum of all LoRA layers output by the gating network; By formula: , the output of the gating network Added to the output of multi-task pre-trained models , get the output of the pre-trained network .

5. The hybrid expert model training method based on LORA technology according to claim 4 is characterized in that: The method further comprises: By formula: ； ； Calculate the total loss function of the multi-task pre-trained model ; in, represents the loss function corresponding to the task, represents a preset constant, and ∈(0,1), represents the true label of the configuration, Represents the preset balancing hyperparameters.

6. The hybrid expert model training method based on LORA technology according to claim 1, characterized in that: Obtain the maximum number of tokens processed by each LoRA layer in a single batch and preset multi-objective optimization strategies to optimize the load balancing of multi-task pre-training model operations, including: By solving the preset multi-objective optimization strategy formula: ； Calculate and obtain the solution set X that reaches the maximum number of iterations or the improvement of the solution set is less than the preset threshold; in, Indicates the variance of the LoRA layer load corresponding to the current solution set X, represents the average system response time corresponding to the current solution set X, Indicates the resource utilization corresponding to the current solution set X, 、 、 Indicates the preset weight coefficient.

7. A hybrid expert model training system based on LORA technology, characterized in that: The system comprises: A pre-trained model preparation module is used to select a program based on preset performance and determine a multi-task pre-trained model that meets the preset performance requirements from a number of multi-task pre-trained models; The dataset construction module is used to create a preset fine-tuning dataset for the Chinese environment corresponding to the tasks in the multi-task pre-training model; The network fine-tuning module is used to train several LoRA layers corresponding to the Transformer block in the multi-task pre-trained model using a preset fine-tuning dataset. During the training process using the preset fine-tuning dataset, a LoRA layer is dynamically selected from several LoRA layers according to the training data corresponding to the current task through a gating network for parameter adjustment. The output of the LoRA layer is adjusted through the gating network to obtain the output of the pre-trained network. The model processing module is used to freeze the parameters of the multi-task pre-training model and the LoRA layer after completing the training of the multi-task pre-training model and the LoRA layer; obtain the maximum number of tokens processed by each LoRA layer in a single batch and preset multi-objective optimization strategy, and perform load balancing optimization for the operation of the multi-task pre-training model; specifically, it includes: The formula for setting the expert capacity threshold is: , calculate the maximum number of tokens processed by the i-th LoRA layer in a single batch ; in, represents the minimum value of the preset capacity threshold of the i-th LoRA layer, represents the maximum value of the preset capacity threshold of the i-th LoRA layer, represents the average of the first k-1 batch requests, Indicates the preset adjustment step size, Indicates the maximum number of tokens processed by the i-th LoRA layer in a single batch before optimization.

8. A hybrid expert model training device based on LORA technology, characterized in that: The device comprises: processor; And a memory having executable code stored thereon, which, when the executable code is executed, enables the processor to execute a hybrid expert model training method based on LORA technology as described in any one of claims 1-6.

9. A non-volatile computer storage medium, characterized in that Computer instructions are stored thereon, and when the computer instructions are executed, they implement a hybrid expert model training method based on LORA technology as described in any one of claims 1 to 6.

Images