Hybrid expert model training system based on layer freezing

Abstract

The invention discloses a hybrid expert model training system based on layer freezing, which relates to the field of model training and comprises an external memory, a freezing controller, a resource allocation controller and a plurality of computing devices. The external memory stores a data set for training the hybrid expert model; the freezing controller performs freezing decision making according to a path selected by an expert and sends the decision to the computing device for execution; the resource allocation controller executes a lexical grouping algorithm based on a frozen layer expert selection path to calculate expert popularity, and allocates computing resources according to the expert popularity; and the computing device stores the hybrid expert model, executes model training operation, executes a layer freezing task according to a layer freezing decision, and feeds back a freezing layer expert selection path and an expert selection path to the resource allocation controller and the freezing controller respectively. According to the method, training of the hybrid expert model is performed through a layer freezing technology, so that the training cost is reduced while the model precision is kept; computing resource allocation is carried out based on a word source grouping algorithm, and equipment load balancing is achieved.

Description

Technical Field

[0001] The present invention relates to the field of model training, and in particular to a hybrid expert model training system based on layer freezing. Background Art

[0002] Distributed training system of Mixture of Experts (MoE): MoE is a large model composed of many small models. Different small models can be considered as experts in different fields and can only be activated when data in their fields are input. Nowadays, MoE has been adapted to different types of deep neural network models, especially showing great potential in improving the performance of language models. In terms of distributed training, in order to adapt to the huge computing requirements of large-scale models, existing systems adopt a variety of hybrid parallel methods such as expert parallelism, data parallelism and tensor parallelism; in order to improve the utilization rate of computing devices, existing systems adopt optimization methods such as adaptive parallelism, shadow experts and heat prediction; in order to reduce all-to-all communication in MoE training, existing systems propose optimization methods such as hierarchical all-to-all algorithms and fine-grained scheduling.

[0003] Layer Freezing Technology: Studies have shown that deep learning models exhibit convergence characteristics from shallow to deep, confirming the feasibility of layer freezing. However, due to the lack of prior knowledge (such as trained models), it is difficult to quantify the training progress of a layer during training. In addition, premature layer freezing can seriously damage the accuracy of the model. Existing layer freezing mechanisms are mostly targeted at traditional deep learning models, using static freezing, gradient indicators, and offline predictors as freezing indicators, which are not suitable for hybrid expert models with dynamic loads. Summary of the invention

[0004] In view of the above problems, the present invention proposes a hybrid expert model training system based on layer freezing. In view of the characteristics of dynamic load and flexible structure of the hybrid expert model, a hybrid expert model training system based on layer freezing technology is proposed, which takes into account both accuracy and efficiency. The present invention utilizes layer freezing technology to freeze the network layer by layer while ensuring the accuracy of the model, thereby reducing the training calculation cost. Through the word unit grouping mechanism, a reasonable resource allocation strategy is calculated to achieve the purpose of equipment load balancing and reducing communication overhead.

[0005] A hybrid expert model training system based on layer freezing includes an external memory, a freezing controller, a resource allocation controller, and several computing devices:

[0006] The external memory stores a data set used to train the hybrid expert model;

[0007] The computing device stores a hybrid expert model including several expert layers, each expert layer including several experts, and the experts represent parameters in the expert layer; the computing device uses the data set to perform a training operation of the hybrid expert model, and during model training, each expert layer of the hybrid expert model selects an expert to perform calculations; the sequence of experts selected by each expert layer constitutes an expert selection path; the computing device freezes the expert layers in order from shallow to deep layers according to the layer freezing decision from the freezing controller, and all frozen layers in the expert layer are frozen layers, and all unfrozen layers are active layers; the computing device transmits the expert selection path to the freezing controller, and transmits the frozen layer expert selection path to the resource allocation controller; the frozen layer expert selection path is the part of the expert selection path located in the frozen layer;

[0008] The freezing controller calculates a freezing evaluation index based on the expert selection path, makes a freezing decision according to the freezing evaluation index, and transmits the freezing decision to each computing device;

[0009] The resource allocation controller executes a word-unit grouping algorithm based on the frozen layer expert selection path, calculates the expert heat, and allocates computing resources of the computing device according to the expert heat.

[0010] Preferably, the freezing evaluation index is calculated based on the expert selection path, and the freezing decision is made according to the freezing evaluation index. The specific steps are as follows:

[0011] The freeze controller collects and stores historical data of expert selection paths from a computing device;

[0012] The freezing controller calculates the freezing evaluation index of the shallowest active layer according to the historical data of the expert selection path. If the freezing evaluation index of the shallowest active layer is less than the preset freezing evaluation index, the freezing evaluation index of a deeper active layer is calculated and compared with the preset freezing evaluation index. This step is repeated until the freezing evaluation index of a certain active layer is greater than or equal to the preset freezing evaluation index, and all active layers before this layer are expert layers that need to be frozen; the shallowest active layer is the expert layer closest to the freezing layer among the active layers.

[0013] Preferably, the frozen evaluation index is the expert selection path change rate; the expert selection path change rate is the ratio of the input data with unchanged expert selection between adjacent iterations to all input data during the training of the hybrid expert model.

[0014] Preferably, the frozen layer expert selection path is based on executing a word-unit grouping algorithm, calculating the expert heat, and allocating computing resources of a computing device according to the expert heat, as follows:

[0015] Divide the expert selection paths of the first i frozen layers into t groups, calculate the expert heat of the next layer, that is, the proportion of the j-th group of words in a batch of the data set, and obtain the heat of the expert e selected by the j-th group of words Then the resource allocation of expert e is expressed as Where N represents the number of computing devices, N t Indicates the number of tokens in a data batch.

[0016] Preferably, the freezing controller calculates the freezing evaluation index and makes a freezing decision in a periodic manner; the period is a number of iterations of the hybrid expert model training.

[0017] Preferably, the computing device includes a computing device memory for caching intermediate results in the hybrid expert model training process and loading the intermediate results into the hybrid expert model to participate in model training.

[0018] Preferably, the intermediate results in the hybrid expert model training process are cached and loaded into the hybrid expert model to participate in model training, as follows:

[0019] Before the model training iteration, the hybrid expert model is sampled in small batches using the data of the next iteration, and the forward calculation results of the frozen layer in the computing device are stored in the external memory as intermediate results;

[0020] The freeze controller pre-fetches the intermediate results in the external memory into the memory of the computing device;

[0021] During the iterative process of model training, the computing device loads the intermediate results from the computing device memory as the forward calculation results of the frozen layer.

[0022] Preferably, the intermediate result is the activation function value of the frozen layer.

[0023] Preferably, the freezing controller runs on a CPU; the calculation of the freezing evaluation index and the making of the freezing decision are performed when the CPU load is lower than a preset load threshold.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] (1) The present invention presents a convergence characteristic from shallow to deep layers according to the expert selection of the hybrid expert model, and proposes a layer freezing technology based on the hybrid expert model, that is, using the expert selection path change rate as the model convergence indicator. While maintaining the model accuracy, the back propagation calculation and all-to-all communication are reduced through expert layer freezing.

[0026] (2) The present invention proposes a word-unit grouping algorithm for load balancing and reducing communication. That is, after each layer freeze, the expert heat is calculated based on the word-unit grouping algorithm, and the expert device mapping is updated to ensure device load balancing and minimize parameter exchanges between devices to reduce all-to-all communication.

[0027] (3) The present invention pre-fetches intermediate results in external memory, caches them in the computing device and participates in model training, thereby skipping the forward calculation of the frozen layer during the training process and effectively overlapping the slower hard disk access with the pre-fetch.

[0028] (4) When the present invention freezes the execution layer, the calculation of the freezing evaluation index and the freezing decision are performed when the CPU is under low load, so as to avoid interference with other CPU-based auxiliary operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The present invention is further described in detail below in conjunction with the accompanying drawings;

[0030] Figure 1 A system architecture diagram of a hybrid expert model training system based on layer freezing according to an embodiment of the present invention;

[0031] Figure 2 A controller-worker framework for a layer-freezing-based hybrid expert model training system according to an embodiment of the present invention;

[0032] Figure 3 A schematic diagram of a freezing evaluation mechanism of a hybrid expert model training system based on layer freezing according to an embodiment of the present invention;

[0033] Figure 4 A schematic diagram of a resource allocation mechanism of a hybrid expert model training system based on layer freezing according to an embodiment of the present invention;

[0034] Figure 5 A schematic diagram of a cache pre-fetching mechanism of a hybrid expert model training system based on layer freezing according to an embodiment of the present invention; DETAILED DESCRIPTION

[0035] The present invention is further described below through specific implementation modes.

[0036] like Figure 1 As shown, the layer-freezing-based hybrid expert model training system of this embodiment includes an external memory, a controller, and several computing devices.

[0037] External memory, storing data sets and intermediate results.

[0038] Controllers, including freezing controllers and resource allocation controllers.

[0039] A computing device stores a hybrid expert model including several expert layers, each expert layer including several experts, and the experts represent parameters in the expert layer; the computing device uses a data set to perform training operations on the hybrid expert model, and during model training, each expert layer of the hybrid expert model selects an expert for calculation; a sequence of experts selected by each expert layer constitutes an expert selection path; the computing device freezes the expert layers in order from shallow to deep layers (the expert layer close to the data input is the shallow layer, and the expert layer close to the model output is the deep layer) according to a layer freezing decision from a freezing controller, and all frozen layers in the expert layer are frozen layers, and all unfrozen layers are active layers; the computing device transmits the expert selection path to the freezing controller, and transmits the frozen layer expert selection path to the resource allocation controller; the frozen layer expert selection path is the part of the expert selection path located in the frozen layer.

[0040] For example, there is a MoE model with only three layers. For an input, expert 3 is selected in the first layer, expert 1 is selected in the second layer, and expert 2 is selected in the third layer. The expert selection path is "expert 3->expert 1->expert 2".

[0041] The freezing controller collects the expert selection paths from each computing device, calculates the freezing evaluation index based on the active layer expert selection path, makes a freezing decision according to the freezing evaluation index, and transmits the freezing decision to each computing device.

[0042] The resource allocation controller collects the frozen layer expert selection paths from each computing device, executes the word unit grouping algorithm based on the frozen layer expert selection paths, and updates the expert device mapping according to the results of the word unit grouping algorithm, that is, the expert heat; the expert device mapping allocates the computing resources of the computing device according to the expert heat of different experts to balance the computing load.

[0043] In this embodiment, the freeze controller and the resource controller are set to make decisions in the same device of the cluster to reduce the overall computing overhead in the case of distributed training.

[0044] See also Figure 2 As shown in the figure, a freezing controller and several computing devices form a controller-worker framework, which is used to perform efficient freezing evaluation. Specifically, the computing device includes forward() method, freeze() method and unfreeze() method. The forward() method performs the forward propagation of the hybrid expert model in the computing device and obtains the intermediate calculation results of the forward propagation. The freeze() and unfreeze() methods respectively perform layer freezing operations, which are called by the freezing controller and applied to the target layer. See Figure 3 As shown, the efficient freezing evaluation mechanism of this embodiment is specifically as follows:

[0045] This embodiment uses the expert selection path change rate as the convergence indicator, that is, the smaller the expert selection path change between iterations, the closer the MoE expert layer is to convergence. Since the efficient freezing evaluation is performed regularly (multiple data batches form an iteration, and the efficient freezing evaluation is performed every few iterations, for example, every 5 / 10 iterations) and runs asynchronously, it will not interfere with other tasks on the CPU and can be well parallelized with the GPU. The specific steps are:

[0046] 1. The freeze controller process collects expert selection paths in the computing device and performs calculations of freeze evaluation indicators and freeze evaluation decisions when the CPU load is low (e.g., less than a preset load threshold of 50%) to avoid interference with other CPU-based auxiliary operations.

[0047] 2. At the same time, the computing device communicates the expert selection path to the frozen controller and continues training, looping without blocking.

[0048] 3. The freezing controller saves the expert selection path of each data batch, and calculates the expert selection path change rate of the shallowest active layer based on historical information to make a freezing decision; if the expert selection change rate of the shallowest active layer is less than the preset expert selection change rate, the expert selection change rate of the deeper active layer is calculated and compared with the preset expert selection change rate, and this step is repeated until the expert selection change rate of a certain active layer is greater than or equal to the preset expert selection change rate, and all active layers before this layer are expert layers that need to be frozen.

[0049] Among them, the shallowest active layer is the expert layer closest to the frozen layer in the active layer, and the expert selection path change rate refers to the proportion of inputs whose expert selection remains unchanged between adjacent iterations to all inputs.

[0050] See also Figure 4 As shown, the computing resource allocation process of the resource allocation controller in this embodiment is specifically as follows:

[0051] After each new layer freezing task, the resource allocation controller is responsible for collecting the frozen layer expert selection paths of the updated word units, calculating the expert heat through the word unit grouping algorithm, and updating the expert device mapping according to the expert heat to balance the computing load. When all resource allocation communications are completed, that is, when the expert parameter (weight) exchange is completed, the model calculation restarts (that is, a new round of batch data is input);

[0052] Specifically, before the first layer freezing and expert heat calculation, computing resources are evenly distributed according to the traditional method; when the layer freezing expert selection path length is less than the specified hyperparameter (the preset expert path length, which is 3 / 6 in this embodiment), heat prediction is not performed.

[0053] Specifically, expert heat refers to the proportion of inputs that select each expert, that is, the amount of computing allocated to each expert, which is used as an indicator of resource allocation. Based on the finding that "similar word units tend to select similar experts", the word unit grouping algorithm groups inputs with the same expert selection path into the same group, so the expert heat can be calculated based on the proportion of each word unit grouping in each data batch. A word unit is an input, and due to the rule that "similar word units tend to select similar experts", word units with the same expert selection path can be grouped together. For example, according to the expert selection paths of the first i frozen layers being divided into t groups, the expert heat of the next layer can be calculated, that is, the proportion of the jth group of word units in the batch, which is the heat of expert e selected by this group of word units. The resource allocation can be calculated Where N is the number of computing devices, N t is the number of tokens in the data batch.

[0054] Expert parallelism assigns different experts to different computing devices. Expert device mapping refers to the expert allocation strategy. Expert heat can be used to balance the computing load by allocating more computing resources to hot experts. When updating expert device mapping, expert parameter communication, that is, communication between computing devices, is performed.

[0055] See also Figure 5 As shown, this embodiment uses caching and prefetching to skip forward calculations. Specifically, before iteration, future mini-batches are sampled in advance (i.e., mini-batch sampling), the forward calculation results of the frozen layers are saved to an external hard disk, and the relevant intermediate results are prefetched into the memory of the computing device so that the active layers can immediately read them as inputs. In the early training stage, if the forward pass of several layers is faster, prefetching can be disabled. In this way, the forward calculation of the frozen layers can be skipped, and the slower hard disk access can be effectively overlapped with the prefetching.

[0056] The intermediate calculation results of the frozen layer are saved to the external hard disk. When performing shallow calculations, the deep layer pre-fetches the relevant intermediate calculation results to the GPU memory. When performing its calculations, it can directly read these intermediate calculation results from the GPU memory as input, thereby skipping the forward calculation of the frozen layer. Among them, the intermediate results are activation values.

[0057] This embodiment presents a convergence characteristic from shallow to deep layers according to the expert selection of the hybrid expert model, and proposes a layer freezing technology based on the hybrid expert model, that is, using the expert selection path change rate as the model convergence indicator. While maintaining the model accuracy, the back propagation calculation and all-to-all communication are reduced through expert layer freezing.

[0058] Based on the finding that "similar words tend to select similar experts", this embodiment proposes a word grouping algorithm for load balancing and reducing communication. That is, after each layer freeze, the expert heat is calculated based on the word grouping algorithm, and the expert device mapping is updated to ensure device load balancing, and minimize parameter exchanges between devices to reduce all-to-all communication.

[0059] The above is only a specific implementation of the present invention, but the design concept of the present invention is not limited thereto. Any non-substantial changes to the present invention using this concept shall be deemed as an infringement of the protection scope of the present invention.

Claims

1. A hybrid expert model training system based on layer freezing, characterized in that: Including external memory, freeze controller, resource allocation controller and several computing devices: The external memory stores a data set used to train the hybrid expert model; The computing device stores a hybrid expert model including a plurality of expert layers, each expert layer including a plurality of experts, and the experts represent parameters in the expert layer; The computing device uses the data set to perform a training operation of the hybrid expert model. During model training, each expert layer of the hybrid expert model selects an expert to perform calculations; the sequence of experts selected by each expert layer constitutes an expert selection path; The computing device freezes the expert layers in order from shallow layers to deep layers according to the layer freezing decision from the freezing controller, and all frozen layers in the expert layers are frozen layers, and all unfrozen layers are active layers; the computing device transmits the expert selection path to the freezing controller, and transmits the frozen layer expert selection path to the resource allocation controller; the frozen layer expert selection path is the part of the expert selection path located in the frozen layer; The freezing controller calculates a freezing evaluation index based on the expert selection path, makes a freezing decision according to the freezing evaluation index, and transmits the freezing decision to each computing device; The resource allocation controller executes a word-unit grouping algorithm based on the frozen layer expert selection path, calculates the expert heat, and allocates computing resources of the computing device according to the expert heat.

2. The hybrid expert model training system based on layer freezing according to claim 1, characterized in that: The freezing evaluation index is calculated based on the expert selection path, and the freezing decision is made according to the freezing evaluation index. The specific steps are as follows: The freeze controller collects and stores historical data of expert selection paths from a computing device; The freezing controller calculates the freezing evaluation index of the shallowest active layer according to the historical data of the expert selection path. If the freezing evaluation index of the shallowest active layer is less than the preset freezing evaluation index, the freezing evaluation index of a deeper active layer is calculated and compared with the preset freezing evaluation index. This step is repeated until the freezing evaluation index of a certain active layer is greater than or equal to the preset freezing evaluation index, and all active layers before this layer are expert layers that need to be frozen; the shallowest active layer is the expert layer closest to the freezing layer among the active layers.

3. The hybrid expert model training system based on layer freezing according to claim 2 is characterized in that: The frozen evaluation index is the expert selection path change rate; the expert selection path change rate is the ratio of the input data with unchanged expert selection between adjacent iterations to all input data during the training of the hybrid expert model.

4. The hybrid expert model training system based on layer freezing according to claim 1, characterized in that: The frozen layer expert selection path is based on executing the word unit grouping algorithm, calculating the expert heat, and allocating the computing resources of the computing device according to the expert heat, as follows: Divide the expert selection paths of the first i frozen layers into t groups, calculate the expert heat of the next layer, that is, the proportion of the j-th group of words in a batch of the data set, and obtain the heat P of the expert e selected by the j-th group of words j i+1 (e), then the resource allocation of expert e is expressed as Where N represents the number of computing devices, N t Indicates the number of tokens in a data batch.

5. The hybrid expert model training system based on layer freezing according to claim 1, characterized in that: The freezing controller calculates the freezing evaluation index and makes freezing decisions in a cycle; the cycle is a number of iterations of the hybrid expert model training.

6. The hybrid expert model training system based on layer freezing according to claim 1, characterized in that: The computing device includes a computing device memory, which caches intermediate results in the hybrid expert model training process and loads the intermediate results into the hybrid expert model to participate in model training.

7. The hybrid expert model training system based on layer freezing according to claim 6, characterized in that: The intermediate results of the hybrid expert model training process are cached and loaded into the hybrid expert model to participate in model training, as follows: Before the model training iteration, the hybrid expert model is sampled in small batches using the data of the next iteration, and the forward calculation results of the frozen layer in the computing device are stored in the external memory as intermediate results; The freeze controller pre-fetches the intermediate results in the external memory into the memory of the computing device; During the iterative process of model training, the computing device loads the intermediate results from the computing device memory as the forward calculation results of the frozen layer.

8. The hybrid expert model training system based on layer freezing according to claim 7, characterized in that: The intermediate result is the activation function value of the frozen layer.

9. The hybrid expert model training system based on layer freezing according to claim 1, characterized in that: The freezing controller runs on the CPU; the calculation of the freezing evaluation index and the making of the freezing decision are performed when the CPU load is lower than a preset load threshold.

Images