Hybrid expert model training system based on layer freezing

Abstract

This invention discloses a hybrid expert model training system based on layer freezing, relating to the field of model training. It includes external memory, a freezing controller, a resource allocation controller, and several computing devices. The external memory stores the dataset used to train the hybrid expert model. The freezing controller makes freezing decisions based on expert selection paths and delegates them to the computing devices for execution. The resource allocation controller calculates expert popularity using a word grouping algorithm based on the expert selection paths of the frozen layers and allocates computing resources according to the expert popularity. The computing devices store the hybrid expert model, perform model training operations, execute layer freezing tasks based on layer freezing decisions, and feed back the expert selection paths of the frozen layers and the expert selection paths to the resource allocation controller and the freezing controller, respectively. This invention trains hybrid expert models using layer freezing technology, maintaining model accuracy while reducing training costs; it allocates computing resources based on word grouping algorithms to achieve device load balancing.

Description

Technical Field

[0001] This invention relates to the field of model training, and more particularly to a hybrid expert model training system based on layer freezing. Background Technology

[0002] Distributed training systems for Mixture of Experts (MoE) models: MoE is a large model composed of many smaller models, which can be considered as experts in different domains, activating only when inputted with data from their respective domains. Currently, MoE has adapted to different types of deep neural network models, showing great potential, particularly in improving the performance of language models. In terms of distributed training, to accommodate the massive computational demands of large-scale models, existing systems employ various hybrid parallel methods such as expert parallelism, data parallelism, and tensor parallelism. To improve the utilization of computing resources, existing systems use optimization methods such as adaptive parallelism, shadow experts, and heat prediction. To reduce all-to-all communication during MoE training, existing systems propose hierarchical all-to-all algorithms and fine-grained scheduling optimization methods.

[0003] Layer Freezing: Research shows that deep learning models exhibit convergence characteristics from shallow to deep, confirming the feasibility of layer freezing. However, due to the lack of prior knowledge (such as a trained model), it is difficult to quantify the training progress of a layer during training. Furthermore, premature layer freezing can severely impair model accuracy. Existing layer freezing mechanisms are mostly designed for traditional deep learning models, using static freezing, gradient metrics, and offline predictors as freezing metrics, and are not suitable for dynamically loaded hybrid expert models. Summary of the Invention

[0004] To address the above problems, this invention proposes a hybrid expert model training system based on layer freezing. Considering the dynamic load and flexible structure of hybrid expert models, this invention proposes a hybrid expert model training system based on layer freezing technology that balances accuracy and efficiency. This invention utilizes layer freezing technology to freeze the network layer by layer while ensuring model accuracy, thereby reducing training computation costs. Through a word grouping mechanism, a reasonable resource allocation strategy is calculated to achieve device load balancing and reduce communication overhead.

[0005] A layer-freezing-based hybrid expert model training system includes external storage, a freeze controller, a resource allocation controller, and several computing devices.

[0006] The external storage stores the dataset used to train the hybrid expert model;

[0007] The computing device stores a hybrid expert model comprising several expert layers, each expert layer containing several experts, where each expert represents a parameter within the expert layer. The computing device uses a dataset to perform training operations on the hybrid expert model. During model training, each expert layer of the hybrid expert model selects one expert for computation. The sequence of experts selected by each expert layer constitutes an expert selection path. Based on layer freezing decisions from a freeze controller, the computing device freezes expert layers in order from shallow to deep layers. All frozen layers are considered frozen layers, and all unfrozen layers are considered active layers. The computing device transmits the expert selection path to the freeze controller and the frozen layer expert selection path to the resource allocation controller. The frozen layer expert selection path is the portion of the expert selection path located within the frozen layer.

[0008] The freeze controller calculates freeze evaluation indicators based on expert-selected paths, makes freeze decisions based on the freeze evaluation indicators, and transmits the freeze decisions to each computing device.

[0009] The resource allocation controller performs a word grouping algorithm based on the expert selection path in the frozen layer, calculates expert popularity, and allocates computing resources to the computing devices according to the expert popularity.

[0010] Preferably, the specific steps for calculating the freeze evaluation index based on the expert-selected path and making a freeze decision based on the freeze evaluation index are as follows:

[0011] The freeze controller collects and stores historical data on expert-selected paths from computing devices;

[0012] The freeze controller calculates the freeze evaluation index of the shallowest activity layer based on the historical data of the expert-selected path. If the freeze evaluation index of the shallowest activity layer is less than the preset freeze evaluation index, the freeze evaluation index of the deeper activity layer is calculated and compared with the preset freeze evaluation index. This step is repeated until the freeze evaluation index of a certain activity layer is greater than or equal to the preset freeze evaluation index. All activity layers before this layer are expert layers that need to be frozen. The shallowest activity layer is the expert layer that is closest to the freeze layer among the activity layers.

[0013] Preferably, the freeze evaluation index is the expert choice path change rate; the expert choice path change rate is the proportion of input data in which the expert choice remains unchanged between adjacent iterations during the training of the hybrid expert model.

[0014] Preferably, the step of executing the term grouping algorithm based on the expert selection path of the frozen layer, calculating expert popularity, and allocating computing resources of the computing device according to expert popularity is as follows:

[0015] Divide the expert selection paths of the first i-th frozen layer into t groups, and calculate the expert popularity of the next layer, which is the proportion of the j-th group of words in a batch of the dataset, to obtain the popularity of expert e who selected the j-th group of words. Then the resource allocation amount for expert e is expressed as: Where N represents the number of computing devices, N t This indicates the number of tokens in a data batch.

[0016] Preferably, the freeze controller calculates freeze evaluation indicators and makes freeze decisions periodically; the period is a number of iterations of the hybrid expert model training.

[0017] Preferably, the computing device includes computing device memory, which caches intermediate results during the training process of the hybrid expert model and loads the intermediate results into the hybrid expert model to participate in model training.

[0018] Preferably, the intermediate results during the training of the hybrid expert model are cached and loaded into the hybrid expert model to participate in model training, as follows:

[0019] Before the model training iteration, the hybrid expert model is sampled in small batches using the data from the next iteration, and the forward computation results of the frozen layer in the computing device are stored as intermediate results in external memory.

[0020] The freeze controller prefetches intermediate results from external storage into the computing device's memory;

[0021] During the iterative process of model training, the computing device loads intermediate results from its memory as the forward computation results of the frozen layer.

[0022] Preferably, the intermediate result is the activation function value of the frozen layer.

[0023] Preferably, the freeze controller runs on the CPU; the calculation of freeze evaluation indicators and the making of freeze decisions are performed when the CPU load is below a preset load threshold.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] (1) Based on the convergence characteristics of expert selection in the hybrid expert model from shallow to deep, this invention proposes a layer freezing technique based on the hybrid expert model. That is, the change rate of expert selection path is used as the model convergence index. While maintaining the model accuracy, the backpropagation calculation and all-to-all communication are reduced by expert layer freezing.

[0026] (2) This invention proposes a word grouping algorithm for load balancing and reducing communication. That is, after each layer freeze, the expert heat is calculated based on the word grouping algorithm, and the expert device mapping is updated to ensure device load balancing and minimize parameter exchange between devices to reduce all-to-all communication.

[0027] (3) By prefetching intermediate results in external storage, caching them in the computing device and participating in model training, the present invention can skip the forward computation of the frozen layer during training and effectively overlap the slower hard disk access with prefetching.

[0028] (4) When the execution layer is frozen, the present invention calculates the freezing evaluation index and makes freezing decisions when the CPU is under low load, so as to avoid interfering with other CPU-based auxiliary operations. Attached Figure Description

[0029] The present invention will now be described in further detail with reference to the accompanying drawings;

[0030] Figure 1 This is a system architecture diagram of a hybrid expert model training system based on layer freezing, according to an embodiment of the present invention.

[0031] Figure 2 This is the controller-worker framework of a layer-freezing-based hybrid expert model training system according to an embodiment of the present invention;

[0032] Figure 3 This is a schematic diagram of the freezing evaluation mechanism of the hybrid expert model training system based on layer freezing according to an embodiment of the present invention;

[0033] Figure 4 This is a schematic diagram of the resource allocation mechanism of the hybrid expert model training system based on layer freezing according to an embodiment of the present invention;

[0034] Figure 5 This is a schematic diagram of the cache prefetch mechanism of the hybrid expert model training system based on layer freezing according to an embodiment of the present invention; Detailed Implementation

[0035] The present invention will be further described below through specific embodiments.

[0036] like Figure 1 As shown, the layer-freezing-based hybrid expert model training system of this embodiment includes external storage, a controller, and several computing devices.

[0037] External storage is used to store datasets and intermediate results.

[0038] Controllers include freeze controllers and resource allocation controllers.

[0039] A computing device stores a hybrid expert model containing several expert layers. Each expert layer contains several experts, and each expert represents a parameter in the expert layer. The computing device uses a dataset to perform training operations on the hybrid expert model. During model training, each expert layer of the hybrid expert model selects an expert for computation. The sequence of experts selected by each expert layer constitutes the expert selection path. Based on the layer freezing decision from the freeze controller, the computing device freezes the expert layers in order from shallow to deep (expert layers closer to the data input are shallow layers, and expert layers closer to the model output are deep layers). All frozen layers in the expert layer are called frozen layers, and all unfrozen layers are called active layers. The computing device transmits the expert selection path to the freeze controller and transmits the frozen layer expert selection path to the resource allocation controller. The frozen layer expert selection path is the portion of the expert selection path located in the frozen layer.

[0040] For example, consider a MoE model with only three layers. If an input selects expert 3 in the first layer, expert 1 in the second layer, and expert 2 in the third layer, the expert selection path would be "expert 3 -> expert 1 -> expert 2".

[0041] The freeze controller collects expert-selected paths from various computing devices, calculates freeze evaluation indicators based on the expert-selected paths in the activity layer, makes freeze decisions based on the freeze evaluation indicators, and transmits the freeze decisions to each computing device.

[0042] The resource allocation controller collects the expert selection paths from each computing device in the frozen layer, executes the word grouping algorithm based on the expert selection paths in the frozen layer, and updates the expert device mapping according to the result of the word grouping algorithm, i.e., the expert popularity. The expert device mapping allocates computing resources to computing devices according to the expert popularity of different experts to balance the computing load.

[0043] In this embodiment, the freeze controller and resource controller are configured to make decisions on the same device in the cluster to reduce the overall computational overhead in the case of distributed training.

[0044] See Figure 2 As shown, a controller-worker framework consists of a freeze controller and several computing devices, used for efficient freeze evaluation. Specifically, the computing devices include the `forward()`, `freeze()`, and `unfreeze()` methods. The `forward()` method performs the forward propagation of the hybrid expert model within the computing device and retrieves the intermediate computation results of the forward propagation. The `freeze()` and `unfreeze()` methods perform layer freezing operations, respectively, called by the freeze controller and applied to the target layer. See also... Figure 3 As shown, the efficient freeze evaluation mechanism in this embodiment is as follows:

[0045] This embodiment uses the rate of change of the expert-selected path as the convergence metric; that is, the smaller the change in the expert-selected path between iterations, the closer the expert layer of the MoE is to convergence. Because the efficient freeze evaluation is performed periodically (multiple data batches form an iteration, and efficient freeze evaluation is performed every few iterations, for example, every 5 / 10 iterations) and asynchronously, it does not interfere with other tasks on the CPU and can be well parallelized with the GPU. The specific steps are as follows:

[0046] 1. The freeze controller process collects expert-selected paths from the computing device and performs freeze evaluation metric calculations and freeze evaluation decisions when the CPU load is low (e.g., less than 50% of a preset load threshold) to avoid interfering with other CPU-based auxiliary operations.

[0047] 2. Meanwhile, the computing device communicates the expert-selected path to the freeze controller and continues training in a loop without blocking.

[0048] 3. The freeze controller saves the expert selection path for each data batch. Based on historical information, it calculates the change rate of the expert selection path of the shallowest activity layer to make a freeze decision. If the change rate of the expert selection path of the shallowest activity layer is less than the preset change rate, the change rate of the expert selection path of the deeper activity layer is calculated and compared with the preset change rate. This step is repeated until the change rate of the expert selection path of a certain activity layer is greater than or equal to the preset change rate. All activity layers before that layer are expert layers that need to be frozen.

[0049] Among them, the shallowest active layer is the expert layer closest to the frozen layer in the active layers, and the expert choice path change rate refers to the proportion of all inputs whose expert choice remains unchanged between adjacent iterations.

[0050] See Figure 4 As shown, the resource allocation process of the resource allocation controller in this embodiment is as follows:

[0051] After each new layer freezing task, the resource allocation controller is responsible for collecting the updated expert selection paths for the frozen layer, calculating expert popularity using a word grouping algorithm, and updating the expert device mapping based on the expert popularity to balance the computational load. Once all resource allocation communication is complete, i.e., the exchange of expert parameters (weights) ends, model computation restarts (i.e., a new batch of data is input).

[0052] Specifically, before the first layer freezing and expert heat calculation, computing resources are allocated evenly using traditional methods; when the expert selection path length for layer freezing is less than the specified hyperparameter (the preset expert path length, which is 3 / 6 in this embodiment), heat prediction is not performed.

[0053] Specifically, expert popularity refers to the proportion of inputs selected by each expert, i.e., the computational load allocated to each expert, used as an indicator of resource allocation. Based on the finding that "similar lexical units tend to select similar experts," the lexical grouping algorithm groups inputs with the same expert selection path into the same group. Therefore, expert popularity can be calculated based on the proportion of each lexical unit grouped in each data batch. A lexical unit is an input. Due to the rule that "similar lexical units tend to select similar experts," lexical units with the same expert selection path can be grouped together. For example, based on the expert selection paths of the first i-th frozen layer, they are divided into t groups. The expert popularity of the next layer can be calculated, i.e., the proportion of the j-th group of lexical units in the batch, which is the popularity of expert e selected by that group of lexical units. Then its resource allocation amount can be calculated. Where N is the number of computing devices, N t This represents the number of terms in the data batch.

[0054] Expert parallelism assigns different experts to different computing devices. Expert device mapping refers to the expert allocation strategy, which balances the computing load by allocating more computing resources to popular experts based on their popularity. Updating expert device mapping involves expert parameter communication, i.e., communication between computing devices.

[0055] See Figure 5 As shown, this embodiment uses caching and prefetching to skip forward computation. Specifically, before each iteration, future mini-batch sampling is performed in advance. The forward computation results of the frozen layers are saved to external hard disk, and relevant intermediate results are prefetched into the computing device's memory so that the active layers can immediately read them as input. In the early training phase, if the forward propagation of several layers is faster, prefetching can be disabled. In this way, the forward computation of the frozen layers can be skipped, effectively overlapping slower hard disk access with prefetching.

[0056] Intermediate calculation results of the frozen layer are saved to external hard disk. During shallow layer calculations, relevant intermediate calculation results are prefetched into GPU memory. When performing calculations in the deeper layers, these intermediate calculation results can be directly read from GPU memory as input, thus skipping the forward calculations of the frozen layer. The intermediate results are the activation values.

[0057] Based on the convergence characteristics of expert selection in hybrid expert models from shallow to deep layers, this embodiment proposes a layer freezing technique based on hybrid expert models. That is, the change rate of expert selection path is used as the model convergence index. While maintaining the model accuracy, the expert layer freezing reduces backpropagation computation and all-to-all communication.

[0058] Based on the finding that "similar terms tend to select similar experts", this embodiment proposes a term grouping algorithm for load balancing and reducing communication. Specifically, after each layer freeze, the expert popularity is calculated based on the term grouping algorithm, and the expert device mapping is updated to ensure device load balancing and minimize parameter exchange between devices to reduce all-to-all communication.

[0059] The above are merely specific embodiments of the present invention, but the design concept of the present invention is not limited thereto. Any non-substantial modifications made to the present invention using this concept shall be considered as infringing upon the protection scope of the present invention.

Claims

1. A hybrid expert model training system based on layer freezing, characterized in that, Includes external storage, freeze controller, resource allocation controller, and several computing devices: The external storage stores the dataset used to train the hybrid expert model; The computing device stores a hybrid expert model containing several expert layers, each expert layer containing several experts, and the experts representing the parameters in the expert layer. The computing device uses a dataset to perform training operations on a hybrid expert model. During model training, each expert layer of the hybrid expert model selects an expert to perform computation; the sequence of experts selected by each expert layer constitutes the expert selection path. The computing device freezes the expert layers in order from shallow to deep according to the layer freezing decision from the freeze controller. All frozen layers in the expert layer are frozen layers, and all unfrozen layers are active layers. The computing device transmits the expert selection path to the freeze controller and the frozen layer expert selection path to the resource allocation controller. The frozen layer expert selection path is the part of the expert selection path located in the frozen layer. The freeze controller calculates freeze evaluation indicators based on expert-selected paths, makes freeze decisions based on the freeze evaluation indicators, and transmits the freeze decisions to each computing device. The resource allocation controller performs a word grouping algorithm based on the expert selection path in the frozen layer, calculates expert popularity, and allocates computing resources to the computing devices according to the expert popularity.

2. The hybrid expert model training system based on layer freezing according to claim 1, characterized in that, The specific steps for calculating the freeze assessment index based on the expert-selected path and making a freeze decision based on the freeze assessment index are as follows: The freeze controller collects and stores historical data on expert-selected paths from computing devices; The freeze controller calculates the freeze evaluation index of the shallowest activity layer based on the historical data of the expert-selected path. If the freeze evaluation index of the shallowest activity layer is less than the preset freeze evaluation index, the freeze evaluation index of the deeper activity layer is calculated and compared with the preset freeze evaluation index. This step is repeated until the freeze evaluation index of a certain activity layer is greater than or equal to the preset freeze evaluation index. All activity layers before this layer are expert layers that need to be frozen. The shallowest activity layer is the expert layer that is closest to the freeze layer among the activity layers.

3. The hybrid expert model training system based on layer freezing according to claim 2, characterized in that, The freeze evaluation index is the expert choice path change rate; the expert choice path change rate is the proportion of input data whose expert choices remain unchanged between adjacent iterations during the training of the hybrid expert model.

4. The hybrid expert model training system based on layer freezing according to claim 1, characterized in that, The algorithm for executing word grouping based on the expert selection path in the frozen layer calculates expert popularity and allocates computing resources to the computing device according to the expert popularity, as follows: Divide the expert selection paths of the first i-th frozen layer into t groups, calculate the expert popularity of the next layer, which is the proportion of the j-th group of words in a batch of the dataset, and obtain the popularity P of expert e in selecting the j-th group of words. j i+1 (e), then the resource allocation for expert e is expressed as Where N represents the number of computing devices, N t This indicates the number of tokens in a data batch.

5. The hybrid expert model training system based on layer freezing according to claim 1, characterized in that, The freeze controller calculates freeze evaluation indicators and makes freeze decisions periodically; the period is a number of iterations of training the hybrid expert model.

6. The hybrid expert model training system based on layer freezing according to claim 1, characterized in that, The computing device includes computing device memory, which caches intermediate results during the training process of the hybrid expert model and loads the intermediate results into the hybrid expert model to participate in model training.

7. The hybrid expert model training system based on layer freezing according to claim 6, characterized in that, The intermediate results of the hybrid expert model training process are cached and loaded into the hybrid expert model to participate in model training, as detailed below: Before the model training iteration, the hybrid expert model is sampled in small batches using the data from the next iteration, and the forward computation results of the frozen layer in the computing device are stored as intermediate results in external memory. The freeze controller prefetches intermediate results from external storage into the computing device's memory; During the iterative process of model training, the computing device loads intermediate results from its memory as the forward computation results of the frozen layer.

8. The hybrid expert model training system based on layer freezing according to claim 7, characterized in that, The intermediate result is the activation function value of the frozen layer.

9. The hybrid expert model training system based on layer freezing according to claim 1, characterized in that, The freeze controller runs on the CPU; the calculation of freeze evaluation indicators and the making of freeze decisions are performed when the CPU load is below a preset load threshold.

Images