Expert large model training method based on mixed retention degree

Abstract

The invention discloses an expert large model training method based on a mixed retention degree, and the method comprises the steps: inserting a mixed retention degree layer behind any layer of a selected basic large model, and obtaining an expert large model; setting a mixed retention degree layer; the mixed retention degree layer is composed of M experts, and each expert has a preset parameter; a routing module is introduced into the hybrid retention degree layer, experts are dynamically selected according to input features, and the randomness of routing is increased through Gaussian noise vectors; parameter values of the M experts are set through a setting strategy; and training the hybrid retention degree layer through a task-related loss function and a load balancing loss function, and in the training process, keeping the hybrid retention degree layer in a trainable state while keeping other parts of the expert large model in a frozen state. According to the invention, the designed load balances the loss, so that each expert is fully trained; and the difference of each expert is ensured from the source.

Description

Technical Field

[0001] The present invention relates to the field of large model fine-tuning and mixed expert large model, and in particular to an expert large model training method based on mixed retention. Background Art

[0002] MoE (Mixture of Experts) is a technology in the field of a wide range of large models. The core idea is to use multiple "expert" models to solve multiple tasks. Specifically, for each input token, the routing module generates a probability distribution whose probability value represents the possibility of assigning the token to each expert. The routing strategy is used to select the expert to be activated from N experts. The most commonly used strategy is Top-K routing, which always activates a fixed number of experts for each token. It calculates a score to represent the probability of selecting each expert, and selects the top K experts with the highest scores for activation.

[0003] Mainstream experts are implemented based on fully connected layers, which will significantly increase the number of parameters in the model. Therefore, researchers in the field have introduced low-rank adapters (LoRA), adapters and other structures to replace fully connected layers to reduce the number of parameters. Regardless of the implementation method of experts, there are two problems with existing work in the field: 1) The current number of parameters needs to be further compressed. 2) The current MoE layer adopts a homogeneous design. That is, each expert has the same structure and parameter size. These learnable parameters are difficult to fully train, which is reflected in the high similarity of their parameter content.

[0004] For problem 1), the present invention introduces an efficient exponentiation operation as an expert to minimize the number of expert parameters (each expert has only one parameter). For problem 2), the present invention designs a load balancing loss based on task information retention to ensure that each expert is fully trained. In addition, the expert parameters in the present invention are preset and do not require training, ensuring that each expert is different from the source. Summary of the invention

[0005] The purpose of the present invention is to provide an expert large model training method based on mixed retention to address the shortcomings of existing methods. The method improves the accuracy of large model training with the least parameter cost.

[0006] To achieve the above object, the present invention provides an expert large model training method based on mixed retention, comprising the following steps: (1) Select a basic large model, insert a mixed degree-retention layer after any layer of the selected basic large model, and obtain an expert large model; (2) Setting a hybrid retention layer in the expert model; the hybrid retention layer is composed of M experts, each of which has pre-set parameters; introducing a routing module in the hybrid retention layer, dynamically selecting experts according to input features, and increasing the randomness of routing through a Gaussian noise vector; and setting the parameter values ​​of the M experts by setting strategies; (3) The hybrid retention layer is trained through task-related loss functions and load balancing loss functions; during the training process, the hybrid retention layer remains in a trainable state, while other parts of the expert large model remain in a frozen state.

[0007] Furthermore, the basic large model includes a Transformer model, a convolutional neural network model, a recurrent neural network model, and a long short-term memory network model.

[0008] Furthermore, in step (2), for the i-th expert Z i , the expression is: ; Among them, X is the input token, that is, Token; P i is the parameter of the i-th expert; sign(X) represents the sign of X; the routing module is introduced to select different experts for each input tag, and the expression is: ; Among them, the input label X is a 1*d vector, W r is a learnable M*d matrix; R(X) is a vector of size 1*M, indicating the probability of routing to each expert; Represents a Gaussian noise vector of size 1*M, which is used to increase the randomness of the routing; selects the probability of the top k in the R(X) vector to calculate the eigenvalue after fine-tuning , the expression is: .

[0009] Furthermore, in step (2), the parameter values ​​of the M experts are set by setting a strategy; the strategy includes a geometric growth strategy, an arithmetic growth strategy, a square growth strategy, a Fibonacci growth strategy, and a mixed growth strategy; wherein: The geometric growth strategy is that the parameter values ​​of different experts increase in a fixed ratio; The arithmetic growth strategy is that the parameter values ​​of different experts grow according to fixed values; The square growth strategy is that after each parameter value is magnified n times, its value is the square of a natural number; The Fibonacci growth strategy, that is, each parameter value is the sum of the previous two parameter values; The hybrid growth strategy is to select any one of the above strategies and then copy a part of the parameter values ​​therein so that the parameter values ​​of N experts are repeated.

[0010] Furthermore, in step (3), load balancing means: complex experts are used for difficult tasks, and simple experts are used for simple tasks; Through the load balancing loss function To constrain each Token to use simple experts first, the expression is: ; ; ; Among them, M is the number of experts, F i It represents the proportion of tokens allocated to the i-th expert to all tokens; represents the probability of the i-th expert being selected, T represents the number of tokens, e i represents the i-th expert, E t represents the set of all experts activated by the tth Token; P i represents the parameter of the i-th expert, i.e., the task information retention degree; represents the probability that the t-th Token is routed to the i-th expert.

[0011] To achieve the above objectives, the present invention also provides an expert large model training device based on mixed retention, including one or more processors for implementing the above expert large model training method based on mixed retention.

[0012] To achieve the above-mentioned purpose, the present invention also provides an electronic device, including a memory and a processor, wherein the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the above-mentioned expert large model training method based on hybrid retention.

[0013] To achieve the above objectives, the present invention also provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the above-mentioned expert large model training method based on mixed retention is implemented.

[0014] The beneficial effects of the present invention are as follows: the present invention introduces an efficient exponentiation operation as an expert, which reduces the number of expert parameters to a minimum (each expert has only one parameter), which is less than the number of parameters used as experts in the field using fully connected layers, LoRA, adapters, etc. The present invention designs a load balancing loss based on task information retention to enable each expert to be fully trained; and the expert parameters in the present invention are preset and do not require training, which ensures that each expert has differences from the source, while the experts learned by the existing hybrid expert method in the field are homogenized, which in turn affects the training effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a structural diagram of the method of the present invention; Figure 2 It is a schematic diagram of the structure of the device of the present invention; Figure 3 It is a schematic diagram of an electronic device of the present invention. DETAILED DESCRIPTION

[0016] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0017] The terms used in the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The singular forms "a", "the" and "the" used in the present invention and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0018] It should be understood that although the terms first, second, third, etc. may be used in the present invention to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present invention, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0019] The present invention is described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the features of the following embodiments and implementations can be combined with each other.

[0020] The present invention provides an expert large model training method based on mixed retention, comprising the following steps: (1) Basic large model selection: Select the basic large model and insert the MoP layer after any layer of the basic large model to obtain the expert large model.

[0021] The present invention proposes a mixed retention model (Mixture-of-Preservation, MoP), which is essentially a mixed expert model (Mixture-of-Expert), which aims to adapt the basic large model to different tasks with the least parameter cost. Existing large models are generally based on the Transformer architecture, so the present invention takes the Transformer architecture as an example to illustrate the technical principles of the MoP model. It should be noted that it is also applicable to basic large models of other architectures such as Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), Long Short-Term Memory Network (LSTM), etc.

[0022] Given a Transformer model, it consists of N blocks, such as Figure 1 As shown in the figure, each Block includes two layer normalizations (LayersNorm), a multi-head self-attention layer (MHSA) and a feed-forward network (FFN). Usually FFN is implemented by a fully connected layer. The MoP layer (fine-tuning module) can be inserted after the MHSA layer, but this does not mean that this is the only place to insert the MoP layer. In fact, the MoP layer can be inserted after each layer in the Block, and multiple MoP layers can be inserted into a Block. The specific location to insert the MoP depends on the experimental results.

[0023] (2) Hybrid Preservation Layer (MoP) Construction: This step mainly refines the MoP layer in the expert model output from step (1). Construct a MoP layer consisting of M experts, each of which has pre-set parameters; introduce a routing module to dynamically select experts based on input features, and increase the randomness of routing through Gaussian noise vectors to ensure that unpopular experts have a certain probability of being visited.

[0024] Each MoP layer consists of M experts, each of whom has a preset parameter. The present invention uses P1, P2...P M To represent the parameters of M experts. For the i-th expert Z i, its calculation logic is as follows:

[0025] Among them, X is the input token; P i is a scalar, indicating the parameter of the ith module (i.e., the ith expert). sign(X) indicates the sign of X. represents the absolute value of X. As long as the value of P is pre-set and its value does not change with the change of input (X), the present invention introduces a routing module (Router) to select different experts for each input tag in the following way:

[0026] Among them, the input label X is a 1*d vector, W r is a learnable M*d matrix. R(X) is a vector of size 1*M, which represents the probability of routing to each expert, and is a Gaussian noise vector of size 1*M, which is used to increase the randomness of routing, so that unpopular experts can also be visited with a certain probability. The present invention selects the probability of the top k in the R(X) vector to calculate the eigenvalue after fine-tuning as follows:

[0027] The value of k is 2. In fact, k can be any preset value. , whose value is determined by the experimental results.

[0028] The parameter values ​​of the M experts are set by domain experts. The following are examples of several setting strategies (geometric growth, arithmetic growth, square growth, Fibonacci growth, mixed growth, etc.) provided by the present invention: A. Geometric growth strategy, [0, 0.1, 0.2, 0.4, ..., Max], that is, the parameter values ​​of different experts increase in a fixed ratio. Max represents the maximum value of the parameter, which is set in advance. Note that the parameter value can be any non-negative value.

[0029] B. Arithmetic growth strategy, [0, 0.1, 0.2, 0.3, 0.4..., Max], that is, the parameter values ​​of different experts increase according to fixed values.

[0030] C. Square growth strategy, [0, 0.1, 0.4, 0.9, 1.6, ..., Max], that is, after each parameter value is amplified 10 times, its value is the square of a natural number.

[0031] D. Fibonacci growth strategy, [0, 0.1, 0.1, 0.2, 0.3, 0.5, 0.8, ..., Max], each value is the sum of the previous two values.

[0032] E. Mixed growth strategy, [0, 0.1, 0.1, 0.1, 0.2, 0.2, 0.2, ..., Max], that is, select any AD strategy and then copy some of its values ​​so that there is a certain amount of repetition in the parameter values ​​of the N experts.

[0033] In particular, when the parameter value is 0, for any input, the corresponding output of the expert is 1, indicating that the input at this time no longer has any effect on training, that is, the input is not worth retaining.

[0034] (3) Training loss function construction: During the training process, the MoP layer constructed in step (2) remains trainable, while the rest of the expert large model constructed in step (1) remains frozen. The MoP layer uses the task-related loss function and load balancing loss Two loss functions are used for training.

[0035] The training loss function consists of two parts, namely the loss function related to the training task And the load balancing loss function ,in The specific implementation of depends on the task, such as classification loss or segmentation loss. The introduction of is to ensure that each expert is fully visited and trained. The load balancing loss in the field often only considers the frequency of each expert being visited, and crudely achieves load balancing. However, the present invention believes that ideal load balancing should encourage more complex experts to be used for difficult tasks, and simple experts to be used for simple tasks (i.e., experts who do not extract task information as much as possible). Specifically, under the condition of equal training accuracy, the present invention constrains each Token to use simple experts preferentially through load balancing loss, and the expression is as follows:

[0036]

[0037]

[0038] Among them, T represents the number of Tokens. represents the probability of the i-th expert being selected, represents the probability that the tth Token is routed to the ith expert, M is the number of experts, and e i represents the i-th expert, F i Indicates the proportion of tokens allocated to the i-th expert to all tokens.t P represents the set of all experts activated by the tth Token. i represents the parameter of the i-th expert, that is, the task information retention degree.

[0039] Corresponding to the aforementioned embodiment of the expert large model training method based on mixed retention degree, the present invention also provides an embodiment of the expert large model training device based on mixed retention degree.

[0040] See also Figure 2 The expert large model training device based on mixed retention provided in an embodiment of the present invention includes one or more processors for implementing the expert large model training method based on mixed retention in the above embodiment.

[0041] The embodiment of the expert large model training device based on hybrid retention of the present invention can be applied to any device with data processing capability, and the device with data processing capability can be a device or apparatus such as a computer. The device embodiment can be implemented through software, or through hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of any device with data processing capability in which it is located reading the corresponding computer program instructions in the non-volatile memory into the internal memory for execution. From the hardware level, if Figure 2 As shown, it is a hardware structure diagram of any device with data processing capability where the expert large model training device based on mixed retention degree of the present invention is located, except Figure 2 In addition to the processor, memory, network interface, and non-volatile memory shown, any device with data processing capabilities in which the apparatus in the embodiments is located may also include other hardware, generally based on the actual functions of the device with data processing capabilities, which will not be described in detail.

[0042] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0043] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the present invention. Ordinary technicians in this field can understand and implement it without paying creative work.

[0044] Corresponding to the embodiment of the above-mentioned expert large model training method based on mixed retention, the embodiment of the present application also provides an electronic device, including: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned expert large model training method based on mixed retention. Figure 3 As shown, it is a hardware structure diagram of any device with data processing capability in the expert large model training method based on mixed retention provided by the embodiment of the present application, except Figure 3 In addition to the processor, memory, DMA controller, disk, and non-volatile memory shown, any device with data processing capabilities in which the apparatus in the embodiment is located may also include other hardware, which will not be described in detail, generally based on the actual functions of the device with data processing capabilities.

[0045] Corresponding to the aforementioned embodiment of the expert large model training method based on mixed retention, an embodiment of the present invention also provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the expert large model training method based on mixed retention in the above embodiment is implemented.

[0046] The computer-readable storage medium may be an internal storage unit of any device with data processing capability described in any of the aforementioned embodiments, such as a hard disk or a memory. The computer-readable storage medium may also be any device with data processing capability, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit of any device with data processing capability and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capability, and may also be used to temporarily store data that has been output or is to be output.

[0047] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

[0048] The above embodiments are only used to illustrate the design ideas and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, any equivalent changes or modifications made based on the principles and design ideas disclosed by the present invention are within the protection scope of the present invention.

Claims

1. A method for training a large expert model based on mixed retention, characterized in that: The steps include: (1) Select a basic large model, insert a mixed degree-retention layer after any layer of the selected basic large model, and obtain an expert large model; (2) Setting a hybrid retention layer in the expert model; the hybrid retention layer is composed of M experts, each of which has pre-set parameters; introducing a routing module in the hybrid retention layer, dynamically selecting experts according to input features, and increasing the randomness of routing through a Gaussian noise vector; and setting the parameter values ​​of the M experts by setting strategies; (3) The hybrid retention layer is trained through task-related loss functions and load balancing loss functions; during the training process, the hybrid retention layer remains in a trainable state, while other parts of the expert large model remain in a frozen state.

2. The expert large model training method based on mixed retention according to claim 1 is characterized in that: The basic large model includes a Transformer model, a convolutional neural network model, a recurrent neural network model, and a long short-term memory network model.

3. The expert large model training method based on mixed retention according to claim 1 is characterized in that: In step (2), for the i-th expert Z i , the expression is: ； Among them, X is the input token, that is, Token; P i is the parameter of the i-th expert; sign(X) represents the sign of X; the routing module is introduced to select different experts for each input tag, and the expression is: ； Among them, the input label X is a 1*d vector, W r is a learnable M*d matrix; R(X) is a vector of size 1*M, indicating the probability of routing to each expert; Represents a Gaussian noise vector of size 1*M, which is used to increase the randomness of the routing; selects the probability of the top k in the R(X) vector to calculate the eigenvalue after fine-tuning , the expression is: 。 4. The expert large model training method based on mixed retention according to claim 1 is characterized in that: In the step (2), the parameter values ​​of the M experts are set by setting strategies; the strategies include geometric growth strategy, arithmetic growth strategy, square growth strategy, Fibonacci growth strategy, and mixed growth strategy; wherein: The geometric growth strategy is that the parameter values ​​of different experts increase in a fixed ratio; The arithmetic growth strategy is that the parameter values ​​of different experts grow according to fixed values; The square growth strategy is that after each parameter value is magnified n times, its value is the square of a natural number; The Fibonacci growth strategy, that is, each parameter value is the sum of the previous two parameter values; The hybrid growth strategy is to select any one of the above strategies and then copy a part of the parameter values ​​therein so that the parameter values ​​of N experts are repeated.

5. The expert large model training method based on mixed retention according to claim 3 is characterized in that: In step (3), load balancing means: complex experts are used for difficult tasks, and simple experts are used for simple tasks; Through the load balancing loss function To constrain each Token to use simple experts first, the expression is: ； ； ； Among them, M is the number of experts, F i It represents the proportion of tokens allocated to the i-th expert to all tokens; represents the probability of the i-th expert being selected, T represents the number of tokens, e i represents the i-th expert, E t represents the set of all experts activated by the tth Token; P i represents the parameter of the i-th expert, i.e., the task information retention degree; represents the probability that the t-th Token is routed to the i-th expert.

6. An expert large model training device based on mixed retention, characterized in that: It includes one or more processors for implementing the expert large model training method based on mixed retention as described in any one of claims 1-5.

7. An electronic device comprising a memory and a processor, characterized in that: The memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the expert large model training method based on hybrid retention as described in any one of claims 1-5 above.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the expert large model training method based on hybrid retention degree as described in any one of claims 1 to 5 is implemented.

Images