Assembly line distribution automatic search method based on intelligent agent

Abstract

The invention discloses a pipeline allocation automatic search method based on an intelligent agent, and belongs to the technical field of distributed training, and the method specifically comprises the steps: obtaining the execution time of a target large model on cluster equipment through a performance analysis module, and calculating the communication time of each neural network layer in the target large model; and constructing a large language model, and automatically generating an iterative search assembly line allocation strategy in combination with an experience database. Model training is not needed, assembly line parallelization can be directly applied, and therefore the problems that in the prior art, the solving speed is low, online training is needed, and a new model cannot be directly adapted are solved; according to the method, the parallel training efficiency of the target large model can be improved, the search time is remarkably shortened under the condition of ensuring the search effect, the search time is only increased along with the expansion of the pp, the influence of the total number of layers of the neural network layers of the target large model is almost avoided, and the method has good universality and robustness.

Description

Technical Field

[0001] The invention belongs to the technical field of distributed training, and in particular relates to an automatic pipeline allocation search method based on intelligent agents. Background Art

[0002] As the scale of deep learning models and data sets continues to grow, the single-machine model can no longer meet the needs of super-large model training. Therefore, distributed training and its parallel technology have become the mainstream method to optimize training efficiency.

[0003] In distributed training, pipeline parallelism is an important parallelization strategy that allows the training process of deep learning models to be divided into multiple stages, each of which is executed in parallel on different devices or nodes. This parallelization method can evenly distribute the computing load to multiple devices to improve the overall training efficiency, and is therefore particularly suitable for models with a large number of layers and a large amount of computation.

[0004] However, pipeline parallelism also faces difficulties and challenges: the pipeline requires careful planning and scheduling to avoid processor idleness. Since the workload is carried out in stages, unbalanced task distribution will cause some stages to wait for other stages to complete, causing the device to be idle, thus affecting the overall training efficiency. Summary of the invention

[0005] In response to the problems existing in the existing pipeline parallel technology, the present invention provides an automatic pipeline allocation search method based on intelligent agents, which can improve the parallel training efficiency of the target large model, significantly shorten the search time while ensuring the search effect, and has good versatility and robustness.

[0006] In order to achieve the above purpose, the technical method adopted by the present invention is as follows:

[0007] The automatic search method for pipeline allocation based on intelligent agent includes the following steps:

[0008] Step 1: Using the performance analysis module, obtain the performance analysis results of the target large model including N neural network layers, including the execution time of the i-th, i=1, 2, ..., N-th neural network layer as the execution time of the target large model on the cluster device;

[0009] Step 2: Assuming that the global batch size of the target large model is 1, the user sets the number of pipeline stages M and the micro-batch size, and calculates the activation memory v of the i-th neural network layer according to the network structure, sequence length, hidden layer and micro-batch size of the target large model. i ,i=1,2,...,N;

[0010] Then the communication time e of the i-th neural network layer is calculated i,i=1,2,...,N:

[0011]

[0012] Where b is the bandwidth between any two devices in the cluster;

[0013] Step 3: Build a large language model and set the output data format of the large language model to [p1, p2, ..., p M ], where p j ,j=1,2,...,M is the number of neural network layers included in the j,j=1,2,...,Mth pipeline stage;

[0014] Step 4: Automatically generate an iterative search pipeline allocation strategy, including:

[0015] Step 4.1, set the number of iterations k = 1; concatenate the number of neural network layers, the number of pipeline stages, the micro-batch size, and the execution time and communication time of the i-th neural network layer of the target large model as the input data for the first iteration; construct an experience database, and set the experience database for the first iteration to be empty;

[0016] Step 4.2: In the kth iteration, input data is fed into the large language model, and the output is used as the pipeline allocation strategy A k , and assign the pipeline strategy A k The execution time and communication time of the N neural network layers in the input data are concatenated to obtain the input O of the kth iteration. k ;

[0017] Step 4.3: Assign strategy A to pipeline k Perform an evaluation and determine the evaluation result Q obtained in the kth iteration according to the evaluation formula k ;

[0018] Step 4.4: Construct the experience obtained from the kth iteration as (O k ,A k ,Q k ), search the empirical database with (O k ,A k ,Q k ) closest experience, and compare the retrieved experience with (O k ,A k ,Q k ) are used as the input data for the k+1th iteration. If no matching data is found in the empirical database, k ,A k ,Q k ) is the closest experience, then only (O k ,A k ,Qk ) as the input data for the k+1th iteration; at the same time, (O k ,A k ,Q k ) is added to the experience database;

[0019] Step 4.5: Determine whether the number of iterations k reaches the preset upper limit of the number of iterations, or whether the pipeline allocation strategy obtained by multiple consecutive iterations remains unchanged. If neither is satisfied, set k = k + 1 and return to step 4.2; otherwise, output the pipeline allocation strategy A obtained by the kth iteration. k , as the optimal pipeline allocation strategy.

[0020] Furthermore, the method for obtaining the execution time of each neural network layer in step 1 is specifically as follows:

[0021] First, discard the first n rounds of training data of the target large model; then count the operator duration between the first appearance of the operator of the i, i=1, 2, ..., N neural network layers and the first appearance of the operator of the i+1 neural network layer, as the initial execution time of the i-th neural network layer; for the next m rounds of training data, calculate the average of the initial execution time of the i-th neural network layer, as the execution time of the i-th neural network layer.

[0022] Furthermore, the large language model described in step 3 is constructed through pre-instruction fine-tuning.

[0023] Furthermore, the evaluation formula of step 4.3 is:

[0024]

[0025] Where B is the size of the micro-batch after pipeline parallel processing; max{·} represents the maximum value; e j is the execution time of the jth pipeline stage, that is, the jth pipeline stage includes p j The sum of the execution time of the neural network layers; c j is the communication time between the jth pipeline stage and the j+1th pipeline stage.

[0026] Further, in step 4.4, k ,A k ,Q k ) The closest empirical method to determine is:

[0027] Calculate (O k ,A k ,Q k ) input O k The Euclidean distance between the input O corresponding to each experience in the experience database is used to find the experience corresponding to the minimum Euclidean distance as the input O corresponding to (O k,A k ,Q k ) closest experience.

[0028] Furthermore, the maximum value of N is generally 128; M is an enumeration of powers of 2, and its value does not exceed N; n is generally 20; and the value range of m is 20 to 30.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] The present invention proposes an automatic search method for pipeline allocation based on intelligent agents. It does not require model training and can be directly applied to pipeline parallelism, thereby solving the problems of slow solution speed, online training and inability to directly adapt to new models in the prior art. The present invention can improve the parallel training efficiency of the target large model and significantly shorten the search time while ensuring the search effect. The search time only increases with the expansion of pp (micro-batch size) and is almost unaffected by the total number of neural network layers of the target large model. It has good versatility and robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0032] Figure 1 It is a flowchart of the automatic search method for pipeline allocation based on intelligent agent proposed in Example 1;

[0033] Figure 2 It is a schematic diagram of the principle of the automatic search method for pipeline allocation based on intelligent agent proposed in Example 1;

[0034] Figure 3 This is a schematic diagram of pipeline parallel processing proposed in Example 1. DETAILED DESCRIPTION

[0035] In order to further understand the present invention, preferred embodiments of the present invention are described below in conjunction with examples, but it should be understood that these descriptions are only for further illustrating the features and advantages of the present invention, rather than limiting the claims of the invention.

[0036] Example 1

[0037] In pipeline parallelism, the neural network layers of the target large model are distributed among multiple devices (such as graphics cards), and the training batch is subdivided into smaller micro-batches. Forward and backward propagation are performed in a pipelined manner in these micro-batches. However, unbalanced task distribution can cause some stages to wait for other stages to complete, causing the device to be idle, thus affecting the overall training efficiency.

[0038] Based on the above problems, this embodiment proposes an automatic search method for pipeline allocation based on intelligent agent, the process is as follows: Figure 1 As shown, the principle is as Figure 2 As shown, the following steps are included:

[0039] Step 1. Use the performance analysis module to obtain the performance analysis results of the target large model containing N = 24 neural network layers, including the execution time of the i-th, i = 1, 2, ..., 24 neural network layers, as the execution time of the target large model on the cluster device; the performance analysis module described in this embodiment is specifically a Profiler module.

[0040] Among them, the method for obtaining the execution time of the i-th neural network layer is specifically as follows:

[0041] First, discard the first 10 rounds of training data of the target large model; then count the operator duration between the first appearance of the operator of the i, i=1, 2, ..., 24th neural network layer and the first appearance of the operator of the i+1th neural network layer, as the initial execution time of the i-th neural network layer; for the last 20 rounds of training data, calculate the average initial execution time of the i-th neural network layer as the execution time of the i-th neural network layer.

[0042] Step 2: Assuming that the global batch size of the target large model is 1, the user sets the number of pipeline stages M=4 and the micro-batch size, and calculates the activation memory v of the i-th neural network layer according to the network structure, sequence length, hidden layer and micro-batch size of the target large model. i ,i=1,2,...,24;

[0043] Then the communication time e of the i-th neural network layer is calculated i ,i=1,2,...,24:

[0044]

[0045] In the formula, when the cluster devices are homogeneous, b is the bandwidth between any two devices in the cluster;

[0046] Step 3: Build a large language model for the intelligent agent. Specifically, fine-tune the Llama-7b instruction based on the alpaca dataset and set the output data format of the large language model to [p1, p2, p3, p4], where p j ,j=1,2,3,4 is the number of neural network layers included in the j,j=1,2,3,4th pipeline stage.

[0047] In the pipeline allocation problem, only consecutive neural network layers can be assigned to the same pipeline stage. In particular, the computation of the i-th neural network layer can be reused with the computation of the i+1-th neural network layer. For example, the 1st neural network layer and the 2nd neural network layer can be assigned to one stage, and the execution time can be summed because they can form a valid pipeline stage; however, it cannot be the sum of the 1st neural network layer and the 3rd neural network layer because they are not consecutive and cannot be assigned to the same pipeline stage.

[0048] Example: For a target large model containing N=24 neural network layers, the output data of the large language model can be [5,6,7,6], where the first pipeline stage includes 5 neural network layers, namely the 1st to 5th neural network layers; the second pipeline stage includes 6 neural network layers, namely the 6th to 11th neural network layers; the third pipeline stage includes 7 neural network layers, namely the 12th to 18th neural network layers; the fourth pipeline stage includes 6 neural network layers, namely the 19th to 24th neural network layers.

[0049] Step 4: Automatically generate an iterative search pipeline allocation strategy. The large language model of the intelligent agent continuously learns how to generate the optimal pipeline allocation strategy through proposals and feedback, including:

[0050] Step 4.1, set the number of iterations k = 1; concatenate the number of neural network layers, number of pipeline stages, micro-batch size, and execution time and communication time of the i-th neural network layer of the target large model as input data for the first iteration; build an experience database, and set the experience database of the first iteration to be empty.

[0051] Step 4.2: In the kth iteration, input data is fed into the large language model, and the output is used as the pipeline allocation strategy A k , and assign the pipeline strategy A k The execution time and communication time of the N neural network layers in the input data are concatenated to obtain the input O of the kth iteration. k .

[0052] Step 4.3: Assign strategy A to pipeline k Perform an evaluation and determine the evaluation result Q obtained in the kth iteration according to the evaluation formulak ; The evaluation formula is:

[0053]

[0054] Where B is the size of the micro-batch after pipeline parallel processing; max{·} represents the maximum value; e j is the execution time of the jth pipeline stage, that is, the jth pipeline stage includes p j The sum of the execution time of the neural network layers; c j is the communication time between the jth pipeline stage and the j+1th pipeline stage.

[0055] For different micro-batch size settings, the equipment usually has different utilization rates. Intuitively, the first term of the evaluation formula reflects the laggard effect, which is amplified by the size of the micro-batch B. If B is assumed to be 1, the above formula can be simplified to This is consistent with the actual schedule: a single batch only performs a single forward pass and a single backward pass, and sends or receives gradients or activations.

[0056] Step 4.4: Construct the experience obtained from the kth iteration as (O k ,A k ,Q k ), search the empirical database with (O k ,A k ,Q k ) closest experience, and compare the retrieved experience with (O k ,A k ,Q k ) are used as the input data for the k+1th iteration. If no matching data is found in the empirical database, k ,A k ,Q k ) is the closest experience, then only (O k ,A k ,Q k ) as the input data for the k+1th iteration; at the same time, (O k ,A k ,Q k ) is added to the experience database;

[0057] Among them, with (O k ,A k ,Q k ) The closest empirical method to determine is:

[0058] Calculate (O k ,A k ,Q k ) input O kThe Euclidean distance between the input O corresponding to each experience in the experience database is used to find the experience corresponding to the minimum Euclidean distance as the input O corresponding to (O k ,A k ,Q k ) closest experience.

[0059] Step 4.5: Determine whether the number of iterations k reaches the preset upper limit of the number of iterations, or whether the pipeline allocation strategy obtained after three consecutive iterations remains unchanged. If neither is satisfied, set k = k + 1 and return to step 4.2; otherwise, output the pipeline allocation strategy A obtained after the kth iteration. k , as the optimal pipeline allocation strategy.

[0060] If we assume that all neural network layers have the same execution cost and the cluster is equipped with the same device communication bandwidth, the optimal pipeline allocation strategy is to balance the number of layers.

[0061] However, when there are differences in the execution time and communication time of each pipeline stage, it will cause relatively large pipeline bubbles, such as Figure 3 The ideal situation is to achieve this through a reasonable pipeline allocation algorithm. Figure 3 As shown in the lower part, the optimization effect of model training is achieved by reducing pipeline bubbles during training.

[0062] In traditional pipeline allocation, the optimal pipeline allocation strategy is usually manually designed based on knowledge and experience, but it does not use automatic search methods, which leads to the challenge of difficult design when the model scales up. In traditional search methods, the global space is searched based on dynamic programming or depth-first search to obtain the optimal pipeline allocation strategy, but the previous search experience is wasted and the scope of the search space is increased. Therefore, this embodiment can significantly shorten the search time by setting up an experience database and automatically searching for experience.

[0063] The AMP algorithm (the technology disclosed in 2022, "AMP: Automatically Finding Model Parallel Strategies with Heterogeneity Awareness") is used as a comparison example of this embodiment, which solves the pipeline allocation strategy based on the dynamic programming algorithm.

[0064] By adjusting the total number of neural network layers N of pp and the target large model, the time consumption of the automatic search pipeline allocation strategy of the AMP algorithm and the pipeline allocation automatic search method based on intelligent agents proposed in this embodiment is compared. As shown in Table 1, it can be seen that the time consumption of the automatic search pipeline allocation strategy of the AMP algorithm increases sharply with the expansion of the input data scale (i.e., pp and N), but the time consumption of the automatic search pipeline allocation strategy of this embodiment only increases the search time with the expansion of the pp scale, and is almost unaffected by the expansion of the N scale. This embodiment has obvious advantages for large-scale target large models. For example, when N=96, the AMP algorithm takes 27838 seconds, about 7.7 hours, while this embodiment only takes 37 seconds.

[0065] Table 1

[0066] N＝24 N＝32 N＝48 N＝64 N＝96 This embodiment pp=2 1.29 4.27 32.45 160.99 1659.89 2.07 pp=4 2.55 12.48 91.45 472.85 4795.99 20.38 pp=8 3.44 26.32 212.49 1027.25 9571.54 30.92 pp=16 4.99 43.97 402.61 1957.95 27838.39 37.99

[0067] A large number of examples are used to compare the quality of the automatic search pipeline allocation strategy of the AMP algorithm and the pipeline allocation automatic search method based on intelligent agent proposed in this embodiment, as shown in Table 2. It can be seen that compared with the AMP algorithm, this embodiment has made a trade-off between search time and search effect. The error of the pipeline allocation strategy of this embodiment is concentrated at about 1%, indicating that the quality of the automatic search pipeline allocation strategy of this embodiment is very close to the AMP algorithm using dynamic programming, that is, the search time is significantly shortened while ensuring the search effect.

[0068] Table 2

[0069]

[0070] The prior art uses dynamic programming algorithms or online training reinforcement learning algorithms to solve pipeline allocation strategies, which face the problems of long solution time and inability to migrate the trained strategy model. This embodiment uses an automatic pipeline allocation search method based on intelligent agents, which does not require model training and can directly apply pipeline parallelism. This can solve the problems of slow solution speed, the need for online training, and the inability to directly adapt to new models in the prior art, thereby improving the efficiency of pipeline parallelism.

[0071] The above embodiments are intended to provide a better understanding of the present invention, but are not limited to the optimal implementation mode and do not constitute a limitation on the content and protection scope of the present invention. Any product identical or similar to the present invention obtained by anyone under the inspiration of the present invention or by combining the present invention with the features of other prior arts shall be within the protection scope of the present invention.

Claims

1. An automatic search method for pipeline allocation based on intelligent agent, characterized in that: The following steps are involved: Step 1: Using the performance analysis module, obtain the performance analysis results of the target large model including N neural network layers, including the execution time of the i-th, i=1, 2, ..., N-th neural network layer as the execution time of the target large model on the cluster device; Step 2: Assuming that the global batch size of the target large model is 1, the user sets the number of pipeline stages M and the micro-batch size, and calculates the activation memory v of the i-th neural network layer according to the network structure, sequence length, hidden layer and micro-batch size of the target large model. i , i=1,2,...,N, and then calculate the communication time e of the i-th neural network layer i ,i=1,2,...,N: Step 3: Build a large language model and set the output data format of the large language model to [p1, p2, ..., p M ], where p j ,j=1,2,...,M is the number of neural network layers included in the j,j=1,2,...,Mth pipeline stage; Step 4: Automatically generate an iterative search pipeline allocation strategy, including: Step 4.1, set the number of iterations k = 1; concatenate the number of neural network layers, the number of pipeline stages, the micro-batch size, and the execution time and communication time of the i-th neural network layer of the target large model as the input data for the first iteration; construct an experience database, and set the experience database for the first iteration to be empty; Step 4.2: In the kth iteration, input data is fed into the large language model, and the output is used as the pipeline allocation strategy A k , and assign the pipeline strategy A k The execution time and communication time of the N neural network layers in the input data are concatenated to obtain the input O of the kth iteration. k ; Step 4.3: Assign strategy A to pipeline k Perform an evaluation and determine the evaluation result Q obtained in the kth iteration according to the evaluation formula k ; Step 4.4: Construct the experience obtained from the kth iteration as (O k ,A k ,Q k ), search the empirical database with (O k ,A k ,Q k ) the closest experience, and compare the retrieved experience with (O k ,A k ,Q k ) are used as the input data for the k+1th iteration. If no matching data is found in the empirical database, k ,A k ,Q k ) is the closest experience, then only (O k ,A k ,Q k ) as the input data for the k+1th iteration; at the same time, (O k ,A k ,Q k ) is added to the experience database; Step 4.5: Determine whether the number of iterations k reaches the preset upper limit of the number of iterations, or whether the pipeline allocation strategy obtained by multiple consecutive iterations remains unchanged. If neither is satisfied, set k = k + 1 and return to step 4.2; otherwise, output the pipeline allocation strategy A obtained by the kth iteration. k , as the optimal pipeline allocation strategy.

2. The automatic search method for pipeline allocation based on intelligent agent according to claim 1, characterized in that: The specific method for obtaining the execution time of each neural network layer in step 1 is: First, discard the first n rounds of training data of the target large model; then count the duration of operators from the first appearance of the operator of the i, i=1, 2, ..., N neural network layers to the first appearance of the operator of the i+1 neural network layer as the initial execution time of the i neural network layer; For the last m rounds of training data, the average of the initial execution time of the i-th neural network layer is calculated as the execution time of the i-th neural network layer.

3. The automatic search method for pipeline allocation based on intelligent agent according to claim 1 is characterized in that: The large language model described in step 3 is constructed through pre-instruction fine-tuning.

4. The automatic search method for pipeline allocation based on intelligent agent according to claim 1 is characterized in that: The evaluation formula for step 4.3 is: Where B is the size of the micro-batch after pipeline parallel processing; max{·} represents the maximum value; e j is the execution time of the jth pipeline stage, that is, the jth pipeline stage includes p j The sum of the execution time of the neural network layers; c j is the communication time between the jth pipeline stage and the j+1th pipeline stage.

5. The automatic search method for pipeline allocation based on intelligent agent according to claim 1 is characterized in that: In step 4.4, k ,A k ,Q k ) The closest empirical method to determine is: Calculate (O k ,A k ,Q k ) input O k The Euclidean distance between the input O corresponding to each experience in the experience database is used to find the experience corresponding to the minimum Euclidean distance as the input O corresponding to (O k ,A k ,Q k ) closest experience.

Images