Model parameter updating method and device, storage medium and computer equipment

Abstract

The invention provides a model parameter updating method and device, a storage medium and computer equipment, and the method comprises the steps: carrying out the iterative training of a target model in a streamline parallel mode, dividing the optimizer state in each stage into a first optimizer state, a second optimizer state and a dynamic optimizer state based on linear programming, and enabling the first optimizer state, the second optimizer state and the dynamic optimizer state in each iteration, when parameter updating is triggered, for each stage, the dynamic optimizer state of the stage is loaded to the GPU so as to execute parameter updating in the GPU, the dynamic optimizer state of the stage is unloaded to the CPU, and updated parameters in the CPU are loaded to the GPU so as to generate model parameters of the stage. According to the method, the dynamic optimizer state is set to be the optimizer state capable of being dynamically loaded between the GPU and the CPU, so that the dynamic optimizer state can be loaded to the GPU to execute parameter updating when the GPU is idle, the utilization rate of the GPU in the parameter updating stage is improved, and then the training efficiency of the model is improved.

Description

Technical Field

[0001] The present application relates to the field of deep learning technology, and in particular to a model parameter updating method, apparatus, storage medium and computer equipment. Background Art

[0002] As the parameters of large language models grow from billions to trillions, traditional single-device training methods are limited by computing resources, resulting in a continuous decrease in training efficiency. To speed up the model training process, parallel strategies can be used to distribute tasks to multiple GPUs for model training. At this time, memory limitations become the main obstacle to further expanding large language models.

[0003] When training large language models on a single or a few multi-GPU nodes, optimizer state is the main contributor to memory usage. A common method to reduce optimizer memory usage is optimizer offloading, which transfers all optimizer states and corresponding update tasks to the CPU. At this time, the CPU's computing resources and memory resources can be used to support large model training, thereby significantly reducing GPU memory usage. However, due to the limited computing power of the CPU, using optimizer offloading to train large language models will result in reduced model training efficiency. Summary of the invention

[0004] The purpose of this application is to solve at least one of the above technical defects, especially the prior art that transfers all or part of the optimizer state and the corresponding update tasks to the CPU. In this case, the CPU's computing resources and memory resources can be used to support large model training, thereby significantly reducing GPU memory usage. However, due to the limited computing power of the CPU, the use of the optimizer offloading method for training large language models will lead to the technical defect of reduced model training efficiency.

[0005] In a first aspect, the present application provides a model parameter updating method, the method comprising:

[0006] Iteratively training the target model in a pipeline parallel manner, wherein the target model is divided into multiple stages for training;

[0007] Based on linear programming, the optimizer state used for parameter updating in each stage is divided into a first optimizer state, a second optimizer state and a dynamic optimizer state;

[0008] In each iteration, when a parameter update is triggered, for each stage, a GPU and a CPU corresponding to the stage are determined, and a dynamic optimizer state of the stage is loaded from the CPU to the GPU, so as to perform a parameter update in the GPU based on the second optimizer state and the dynamic optimizer state of the stage;

[0009] Based on the first optimizer state of the stage, updating the parameters corresponding to the first optimizer state of the stage in the CPU;

[0010] The dynamic optimizer state of this stage is unloaded to the CPU, and the parameters updated in the CPU are loaded into the GPU to generate the model parameters of this stage.

[0011] In one embodiment, dividing the optimizer state for parameter updating in each stage into a first optimizer state, a second optimizer state and a dynamic optimizer state based on linear programming includes:

[0012] An objective function is constructed with minimizing the parameter update time of the first stage as the optimization goal, and its corresponding constraint conditions are determined according to the objective function; the objective function uses the first optimizer state, the second optimizer state and the dynamic optimizer state of each stage as decision variables;

[0013] Solving the objective function based on the constraint condition to determine the optimizer state division situation at each stage when the optimization goal is achieved, wherein the optimizer state division situation includes division values ​​of a first optimizer state, a second optimizer state, and a dynamic optimizer state;

[0014] The first optimizer state, the second optimizer state and the dynamic optimizer state in each stage are determined according to the optimizer state division situation of each stage when the optimization target is achieved.

[0015] In one embodiment, the loading of the updated parameters in the CPU into the GPU to generate the model parameters of this stage includes:

[0016] After determining the updated parameters in the CPU and loading the parameters into the GPU, determining the updated first parameter, second parameter, and third parameter in the GPU, wherein the first parameter is a parameter corresponding to the first optimizer state of the stage, the second parameter is a parameter corresponding to the second optimizer state of the stage, and the third parameter is a parameter corresponding to the dynamic optimizer state of the stage;

[0017] The first parameter, the second parameter and the third parameter are fused to obtain the model parameters of this stage.

[0018] In one embodiment, the updating of the parameters corresponding to the first optimizer state of the stage in the CPU based on the first optimizer state of the stage includes:

[0019] Transmitting a first gradient corresponding to a first optimizer state of the stage on the GPU to the CPU;

[0020] Parameters corresponding to the first optimizer state of the stage are updated on the CPU based on the first optimizer state of the stage and the first gradient.

[0021] In one embodiment, performing parameter update in the GPU based on the second optimizer state and the dynamic optimizer state of the stage includes:

[0022] Determining, in the GPU, a second gradient corresponding to a second optimizer state of the stage and a third gradient corresponding to a dynamic optimizer state of the stage;

[0023] receiving a gradient sent by a predecessor stage of the stage, and sending the second gradient to a successor stage of the stage;

[0024] Generate a target gradient according to the gradient sent by the predecessor stage of the stage and the second gradient;

[0025] Based on the second optimizer state of the stage and the target gradient, updating the parameters corresponding to the second optimizer state of the stage on the GPU;

[0026] Based on the dynamic optimizer state of the stage and the third gradient, parameters corresponding to the dynamic optimizer state of the stage are updated on the GPU.

[0027] In one embodiment, updating the parameters corresponding to the second optimizer state of the stage on the GPU based on the second optimizer state of the stage and the target gradient includes:

[0028] Calculate the parameter corresponding to the second optimizer state of this stage according to the second optimizer state of this stage and the target gradient to obtain the first parameter;

[0029] receiving a parameter sent by a successor stage of the stage, and sending the first parameter to a predecessor stage of the stage;

[0030] The parameters corresponding to the second optimizer state of this stage are updated based on the parameters sent by the successor stage of this stage and the first parameters.

[0031] In one of the embodiments, before the step of performing parameter updating in the GPU based on the second optimizer state and the dynamic optimizer state of the stage, the method further includes:

[0032] Determining, in the GPU, a gradient corresponding to a first optimizer state of the stage, a gradient corresponding to a second optimizer state of the stage, and a gradient corresponding to a dynamic optimizer state of the stage;

[0033] The determined gradient is overflow checked to ensure the numerical integrity of the determined gradient.

[0034] In a second aspect, the present application provides a model parameter updating device, the device comprising:

[0035] A model training module, used to iteratively train the target model in a pipeline parallel manner, wherein the target model is divided into multiple stages for training;

[0036] An optimizer partitioning module, used for partitioning the optimizer state used for parameter updating in each stage into a first optimizer state, a second optimizer state and a dynamic optimizer state based on linear programming;

[0037] A first update module is used for, in each iteration, when a parameter update is triggered, for each stage, determining a GPU and a CPU corresponding to the stage, and loading a dynamic optimizer state of the stage from the CPU to the GPU, so as to perform parameter update in the GPU based on a second optimizer state and a dynamic optimizer state of the stage;

[0038] A second updating module, configured to update, in the CPU, parameters corresponding to the first optimizer state of the stage based on the first optimizer state of the stage;

[0039] The parameter generation module is used to unload the dynamic optimizer state of this stage to the CPU, and load the updated parameters in the CPU into the GPU to generate the model parameters of this stage.

[0040] In a third aspect, the present application provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the model parameter updating method described in any of the above embodiments.

[0041] In a fourth aspect, the present application provides a computer device, comprising: one or more processors, and a memory;

[0042] The memory stores computer-readable instructions, and when the one or more processors execute the computer-readable instructions, the steps of the model parameter updating method described in any one of the above embodiments are performed.

[0043] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0044] The present application provides a model parameter updating method, device, storage medium and computer equipment, the method comprising: iteratively training a target model in a pipeline parallel manner, and dividing the target model into multiple stages for training, and then dividing the optimizer state used for parameter updating in each stage into a first optimizer state, a second optimizer state and a dynamic optimizer state based on linear programming, so that in each iteration, when the parameter update is triggered, for each stage, the GPU and CPU corresponding to the stage are determined, and then the dynamic optimizer state of the stage is loaded from the CPU to the GPU, so as to perform parameter update in the GPU based on the second optimizer state and the dynamic optimizer state of the stage, and then based on the first optimizer state of the stage, the parameters corresponding to the first optimizer state of the stage are updated in the CPU, and finally the dynamic optimizer state of the stage is unloaded to the CPU, and the updated parameters in the CPU are loaded into the GPU to generate the model parameters of the stage. By dividing the optimizer state, the dynamic optimizer state is set to an optimizer state that can be dynamically loaded between the GPU and the CPU. In this way, the dynamic optimizer state can be loaded onto the GPU when the GPU is idle to perform parameter updates, avoiding more idle periods of the GPU during the parameter update phase due to unloading the optimizer to the CPU, and improving the utilization of the GPU during the parameter update phase, thereby saving video memory while also improving the training efficiency of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0046] Figure 1 A flow chart of a model parameter updating method provided in an embodiment of the present application;

[0047] Figure 2 A comparative example diagram of model parameter update provided in an embodiment of the present application;

[0048] Figure 3 A flow chart of dividing the optimizer state for parameter updating in each stage into a first optimizer state, a second optimizer state and a dynamic optimizer state based on linear programming provided in an embodiment of the present application;

[0049] Figure 4 An example diagram of an application environment of the model parameter updating method provided in an embodiment of the present application;

[0050] Figure 5A schematic diagram of the structure of a model parameter updating device provided in an embodiment of the present application;

[0051] Figure 6 An internal structure diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0053] In one embodiment, the present application provides a model parameter update method, and the following embodiment is described by applying the method to a server. It can be understood that the model parameter update method can be executed by a single server or a server cluster composed of multiple servers, and the present application does not make specific restrictions on this.

[0054] like Figure 1 As shown, the present application provides a model parameter updating method, the method comprising:

[0055] S101: Iteratively train the target model in a pipeline parallel manner.

[0056] Among them, the target model is divided into multiple stages for training.

[0057] In this step, when the user has the need to train a large language model, due to the large data scale and large parameter magnitude of the large language model, the target model to be trained can be iteratively trained in a pipeline parallel manner. Specifically, the pipeline parallel training method divides the target model into multiple stages along the dimension of the layer, and each stage is assigned to a different computing device (such as a GPU). During the forward propagation process, the data is calculated from the first stage, and the output is passed to the next stage, and so on until the model output is obtained in the last stage. During the back propagation process, the gradient information is also passed back in the order of the stages. Each stage calculates the gradient of the part it is responsible for, and then updates the model parameters responsible for this stage according to the gradient. This parallel method can make full use of computing resources, effectively reduce the overall training time of the model, and show good scalability and efficiency when processing large-scale models and large amounts of data.

[0058] S102: Divide the optimizer state used for parameter updating in each stage into a first optimizer state, a second optimizer state and a dynamic optimizer state based on linear programming.

[0059] The optimizer state refers to the dynamic information maintained by the optimizer in order to effectively update the model parameters during the model training process. The first optimizer state refers to the optimizer state located in the CPU, the second optimizer state refers to the optimizer state located in the GPU, and the dynamic optimizer state refers to the optimizer state dynamically loaded between the GPU and the CPU.

[0060] In this step, a function model of the optimizer partitioning problem can be constructed based on the idea of ​​linear programming, and then the optimal solution of the function model can be solved under certain conditions to determine the first optimizer state, the second optimizer state and the dynamic optimizer state of each stage, and then the first optimizer state, the second optimizer state and the dynamic optimizer state determined in each stage are placed on the CPU, GPU and CPU respectively. It should be noted that the second optimizer state on the GPU of each stage includes the optimizer state of itself and multiple predecessor stages.

[0061] It can be understood that by dividing the optimizer state used for parameter update, the parameter update task is distributed, and since a part of the optimizer state is divided as the dynamic optimizer state, it is possible to balance the GPU video memory and parameter update efficiency for dynamic adjustment, so as to save video memory to increase the upper limit of the model size while saving training time from the parameter update aspect, thereby improving the training efficiency of large models.

[0062] S103: In each iteration, when the parameter update is triggered, for each stage, the GPU and CPU corresponding to the stage are determined, and the dynamic optimizer state of the stage is loaded from the CPU to the GPU to perform parameter update in the GPU based on the second optimizer state and the dynamic optimizer state of the stage.

[0063] Specifically, the condition for triggering parameter update can be set to when back propagation is completed in each stage.

[0064] In each iteration, parameter update is triggered when each stage completes back propagation. For each stage, the GPU and CPU of the device corresponding to the stage are determined, and then the dynamic optimizer state of the stage is loaded from the CPU to the GPU. In this way, a part of the parameter update task can be executed in the GPU based on the second optimizer state and the dynamic optimizer state of the stage.

[0065] It is understandable that the dynamic optimizer state is loaded onto the GPU during the parameter update phase to reduce the idle period of the GPU during the parameter update phase. When the parameter update is completed, the dynamic optimizer state is unloaded from the GPU and loaded onto the CPU, thereby reducing the video memory consumption during the forward and backward propagation phases. Therefore, when the parameter update is triggered, the dynamic optimizer state needs to be loaded from the CPU to the GPU, so that part of the optimizer state (dynamic optimizer state) can be dynamically loaded onto the GPU during the GPU idle period to perform the parameter update, and then unloaded back to the CPU after the parameter update is completed, so as to make full use of the GPU during the parameter update phase and save video memory consumption.

[0066] S104: Based on the first optimizer state of this stage, the parameters corresponding to the first optimizer state of this stage are updated in the CPU.

[0067] In this step, the parameters corresponding to the first optimizer state of this stage are updated by the first optimizer state located on the CPU to obtain the updated parameters in the CPU.

[0068] S105: Unloading the dynamic optimizer state of this stage to the CPU, and loading the updated parameters in the CPU into the GPU to generate the model parameters of this stage.

[0069] When the parameter update is completed, the dynamic optimizer state of this stage can be unloaded back to the CPU, and then the updated parameters in the CPU can be loaded into the GPU to generate the model parameters corresponding to this stage in the GPU.

[0070] In one example, if Figure 2 As shown, Figure 2 This is a comparative example diagram of model parameter update provided in the embodiment of the present application. Figure 2 (a) is an example diagram of the parameter update phase in the existing pipeline parallel training, that is, a single parameter update. Figure 2 (b) is an example diagram of the model parameter update method provided by this application, namely, collaborative parameter update. Specifically, in a pipeline consisting of S stages, due to data dependency, the i-th stage needs to perform si micro-batches of forward propagation in the warm-up phase, accumulating a total of si activation values. Therefore, the memory usage of each stage is uneven. Figure 2 Taking (a) as an example, stage 0 needs to perform 3 forward propagations, while stage 2 only needs to perform 1 forward propagation.

[0071] exist Figure 2In (a), since the optimizer state is offloaded to the CPU to save video memory, most parameter update tasks are completed on the CPU, resulting in different lengths of idle periods on the GPU during the parameter update phase (the bubble area and white area in the figure). This results in low resource utilization and parameter update efficiency during the parameter update phase. Figure 2 In (b), since the pre-divided dynamic optimizer states of the corresponding stages are loaded from the CPU to the CPU, the GPU can also perform part of the parameter update tasks in the parameter update stage, thereby reducing the time consumed by the parameter update as a whole, thereby improving the resource utilization in the parameter update stage, thereby improving the efficiency of the model parameter update and accelerating the model training process.

[0072] The present application provides a model parameter updating method, device, storage medium and computer equipment, the method comprising: iteratively training a target model in a pipeline parallel manner, and dividing the target model into multiple stages for training, and then dividing the optimizer state used for parameter updating in each stage into a first optimizer state, a second optimizer state and a dynamic optimizer state based on linear programming, so that in each iteration, when the parameter update is triggered, for each stage, the GPU and CPU corresponding to the stage are determined, and then the dynamic optimizer state of the stage is loaded from the CPU to the GPU, so as to perform parameter update in the GPU based on the second optimizer state and the dynamic optimizer state of the stage, and then based on the first optimizer state of the stage, the parameters corresponding to the first optimizer state of the stage are updated in the CPU, and finally the dynamic optimizer state of the stage is unloaded to the CPU, and the updated parameters in the CPU are loaded into the GPU to generate the model parameters of the stage. By dividing the optimizer state, the dynamic optimizer state is set to an optimizer state that can be dynamically loaded between the GPU and the CPU. In this way, the dynamic optimizer state can be loaded onto the GPU when the GPU is idle to perform parameter updates, avoiding more idle periods of the GPU during the parameter update phase due to unloading the optimizer to the CPU, and improving the utilization of the GPU during the parameter update phase, thereby saving video memory while also improving the training efficiency of the model.

[0073] like Figure 3 As shown, in one embodiment, the optimizer state for parameter updating in each stage is divided into a first optimizer state, a second optimizer state and a dynamic optimizer state based on linear programming, including:

[0074] S201: construct an objective function with minimizing the parameter update time of the first stage as the optimization goal, and determine the corresponding constraint conditions based on the objective function.

[0075] The objective function uses the first optimizer state, the second optimizer state and the dynamic optimizer state in each stage as decision variables. The constraints include the first constraint, the second constraint, the video memory constraint, the computing time constraint and the communication time constraint.

[0076] In this step, since the parameter update time of other stages except the first stage is limited by the computation time constraint in the constraint condition, only the parameter update time of the bottleneck stage (the first stage) needs to be considered, so as to minimize the parameter update time of the first stage to accelerate the update efficiency of the entire parameter update stage. Based on this, the objective function can be constructed with minimizing the parameter update time of the first stage as the optimization goal, and then the corresponding constraint condition can be determined according to the objective function.

[0077] In an example, the constructed objective function and its corresponding constraints can be expressed as:

[0078]

[0079]

[0080] In the formula, Minimize means minimization, Represents the target function, indicating the first stage on the CPU The parameter update time, st represents the constraint condition, , denote the first constraint and the second constraint respectively, Representation stage The first optimizer state, Representation stage The dynamic optimizer state, Representation stage And stage The second optimizer state is Time indication stage or stage The second optimizer state, represents the total amount of optimizer state for the target model, Representation stage The memory constraints, Representation stage The highest memory usage during the training iterations (including forward propagation, backpropagation, and parameter update phases), Representation stage The highest memory usage during forward and backward propagation, Representation stage The highest video memory usage during the parameter update phase, Indicates the rated memory of the GPU, , is the preset coefficient, Representation stage The computation time constraint is Indicates the CPU stage The parameter update time, represents the forward propagation time of a micro-batch, Indicates the communication channel The communication time constraint, Representation stage The time for loading the dynamic optimizer state for the corresponding channel group, Indicates the communication channel The number of communication packets, Indicates the communication channel The total loading capacity, represents the back-propagation time of a micro-batch, Represents the available loading time. and The calculation expression can be expressed as:

[0081]

[0082]

[0083] In the formula, Representation stage The memory usage of the model parameters, Representation stage The memory usage of a micro-batch.

[0084] Specifically, Used to consider other video memory overhead, its value range is 0 to 1, It is used to consider the relaxation after the release of CPU resources. In addition, in the optimization problem corresponding to the objective function, the decision variable can be expressed as , ,in, and for dimensional vector, for dimensional matrix. Considering that the computing power of GPU is significantly higher than that of CPU, but the video memory is limited, and the communication time between GPU and CPU is long, GPU video memory constraint, CPU computing time constraint and communication time constraint are introduced.

[0085] (1) Memory constraint (GPU memory constraint): During pipeline parallel training, the peak memory consumption of each stage may occur in two places: one is during the pipeline warm-up phase, that is, before the first micro-batch backpropagation is completed; the other is before global synchronization. To prevent training failure due to insufficient memory, the memory consumption of the two peaks is constrained, namely, and .

[0086] (2) Computation time constraint (CPU computation time constraint): To ensure that the parameter update time of each stage is consistent with the instant of its forward propagation (i.e., arithmetic progression), constraints can be imposed on the CPU computation time.

[0087] (3) Communication time constraint: To ensure that the dynamic optimizer states loaded by all GPUs connected to the same PCIe channel do not exceed the channel's transport capacity, a constraint can be imposed on the communication time between the GPU and the CPU so that the loading time is less than or equal to the available loading time.

[0088] S202: Solving the objective function based on the constraint conditions to determine the optimizer state division status at each stage when the optimization target is achieved.

[0089] The optimizer state division situation includes division values ​​of the first optimizer state, the second optimizer state and the dynamic optimizer state.

[0090] In this step, the optimal solution of the objective function under the constraint conditions can be solved using a corresponding solution method to determine the optimizer state division at each stage when the optimization goal is achieved. Specifically, the solution method can use a heuristic algorithm, a branch and bound method, a cutting plane method, etc., and this application does not make specific restrictions on this.

[0091] S203: Determine the first optimizer state, the second optimizer state, and the dynamic optimizer state in each stage according to the division of optimizer states in each stage when the optimization target is achieved.

[0092] When the first optimizer state, the second optimizer state and the dynamic optimizer state of each stage are determined, the first optimizer state of each stage can be placed on the CPU of the corresponding stage, the second optimizer state of each stage can be placed on the GPU of the corresponding stage, and the dynamic optimizer state of each stage can be placed on the CPU of the corresponding stage.

[0093] In this embodiment, due to the imbalance of the GPU in each stage during pipeline training, the GPU usage in the parameter update stage of the pipeline parallelization and optimizer offloading is analyzed, and then the optimization target is determined and the objective function is constructed. Finally, the optimal solution that can meet the optimization target is obtained, and the optimizer state division of each stage is configured in advance, so as to maximize the utilization of the GPU, accelerate the model training process, and improve the model training efficiency.

[0094] In one embodiment, loading the updated parameters in the CPU into the GPU to generate the model parameters of this stage includes:

[0095] S1: After determining updated parameters in the CPU and loading the parameters into the GPU, the updated first parameter, second parameter, and third parameter are determined in the GPU.

[0096] Among them, the first parameter is the parameter corresponding to the first optimizer state of this stage, that is, the updated parameter determined in the CPU, the second parameter is the parameter corresponding to the second optimizer state of this stage, and the third parameter is the parameter corresponding to the dynamic optimizer state of this stage.

[0097] In this step, after the parameter update is completed, the updated parameters in the CPU need to be sent to the GPU. At this time, the GPU has the parameters (second parameters) corresponding to the second optimizer state of this stage after the update and the parameters (third parameters) corresponding to the dynamic optimizer state of this stage. Then, combined with the parameters (first parameters) updated in the CPU, the updated first parameters, second parameters and third parameters can be determined and obtained.

[0098] It should be noted that, in the present application, the transmission between the GPU and the CPU may adopt PCIe full-duplex transmission.

[0099] S2: Perform parameter fusion on the first parameter, the second parameter and the third parameter to obtain the model parameters of this stage.

[0100] Specifically, when the first parameter, the second parameter and the third parameter are collected on the GPU, parameter fusion is performed on the first parameter, the second parameter and the third parameter, and the parameters obtained after the parameter fusion are determined as the model parameters of this stage.

[0101] Furthermore, the process of performing parameter fusion on the first parameter, the second parameter and the third parameter may use a weighted average method, an incremental update fusion method and the like to perform parameter fusion, and the present application does not impose any specific limitation on this.

[0102] In this embodiment, after the parameter update is completed, the dynamic optimizer state is unloaded from the GPU and loaded onto the CPU, which can save GPU video memory. At the same time, the updated parameters in the CPU are transmitted to the GPU to determine the first parameter, the second parameter and the third parameter in the GPU, thereby generating the model parameters of this stage.

[0103] In one embodiment, updating parameters corresponding to the first optimizer state of the stage in the CPU based on the first optimizer state of the stage includes:

[0104] S1: Transfer the first gradient corresponding to the first optimizer state of this stage on the GPU to the CPU.

[0105] Specifically, since both forward propagation and back propagation are completed on the GPU at each stage, the gradient obtained at the end of back propagation at each stage is also stored on the GPU. Since the first optimizer state has been pre-divided on the CPU, the first gradient corresponding to the first optimizer state of this stage on the GPU needs to be transmitted to the CPU to perform parameter update on the CPU based on the first gradient and the first optimizer state.

[0106] S2: updating parameters corresponding to the first optimizer state of the stage on the CPU based on the first optimizer state of the stage and the first gradient.

[0107] In this embodiment, the first gradient corresponding to the first optimizer state is transmitted to the CPU, so that parameter updating is performed on the CPU based on the first gradient and the first optimizer state.

[0108] In one embodiment, performing parameter updating in the GPU based on the second optimizer state and the dynamic optimizer state of the stage includes:

[0109] S1: Determine, in the GPU, a second gradient corresponding to the second optimizer state of the stage and a third gradient corresponding to the dynamic optimizer state of the stage.

[0110] S2: Receive the gradient sent by the predecessor stage of this stage, and send the second gradient to the successor stage of this stage.

[0111] Among them, the prodromal stage refers to the stage before this stage, and the successor stage refers to the stage after this stage.

[0112] In this step, the gradient sent by the predecessor stage can be combined with the gradient calculated by itself during the back propagation process to obtain a complete gradient for updating the model parameters responsible for this stage. Similarly, the second gradient is sent to the successor stage so that the successor stage can also perform similar complete gradient calculation and parameter update.

[0113] S3: Generate a target gradient based on the gradient sent by the predecessor stage of this stage and the second gradient.

[0114] S4: Update the parameters corresponding to the second optimizer state of this stage on the GPU based on the second optimizer state of this stage and the target gradient.

[0115] S5: Based on the dynamic optimizer state of this stage and the third gradient, the parameters corresponding to the dynamic optimizer state of this stage are updated on the GPU.

[0116] In this embodiment, the update of the parameters corresponding to the second optimizer state of this stage is implemented in the GPU through gradient transmission, and because the dynamic optimizer state is loaded on the GPU in the parameter update stage, the update of the parameters corresponding to the dynamic optimizer state of this stage is also performed in the GPU.

[0117] In one embodiment, updating parameters corresponding to the second optimizer state of the stage on the GPU based on the second optimizer state of the stage and the target gradient includes:

[0118] S1: Calculate the parameters corresponding to the second optimizer state of this stage according to the second optimizer state of this stage and the target gradient to obtain the first parameters.

[0119] S2: Receive the parameters sent by the successor stage of this stage, and send the first parameters to the predecessor stage of this stage.

[0120] S3: Update the parameters corresponding to the second optimizer state of this stage based on the parameters sent by the subsequent stage of this stage and the first parameters.

[0121] In this embodiment, since the update of the parameters corresponding to the second optimizer state involves the gradient of the predecessor stage, the first parameters are sent to the predecessor stage of the stage, and since the second gradient corresponding to the second optimizer state of the stage is sent to the successor stage, the parameters sent by the successor stage of the stage are received, thereby completing the update of the parameters corresponding to the second optimizer state of the stage.

[0122] In one embodiment, before the step of performing parameter updating in the GPU based on the second optimizer state and the dynamic optimizer state of the stage, the model parameter updating method further includes:

[0123] S1: Determine in the GPU a gradient corresponding to a first optimizer state of the stage, a gradient corresponding to a second optimizer state of the stage, and a gradient corresponding to a dynamic optimizer state of the stage.

[0124] S2: Perform overflow check on the determined gradient to ensure the numerical integrity of the determined gradient.

[0125] In this embodiment, firstly, from the perspective of model training stability, the check can avoid the training process from getting out of control due to excessive or too small gradient values. When the gradient overflows, the parameter update may be abnormal, such as causing the model parameter to become infinite or non-number (NaN), and the overflow check can detect this potential crisis in time to ensure that the training process can proceed smoothly in the optimization direction. Secondly, in terms of model accuracy, the appropriate gradient value enables the model to adjust parameters based on reasonable gradient information, which helps to improve the accuracy and speed of model convergence.

[0126] In one example, if Figure 4 As shown, Figure 4 This is an example diagram of the application environment of the model parameter updating method provided in the embodiment of the present application. Figure 4 In the embodiment, the user provides a data set, a model (i.e., the target model in the present application), and a training configuration (corresponding to step 1), and then uses a preset analyzer to analyze the above data provided by the user to determine a bubble situation, wherein the bubble situation can reflect the idle time of the GPU, and then sends the bubble situation to a preset bubble scheduler (corresponding to step 2). At this time, the bubble scheduler can determine the optimizer state division of each stage according to the bubble situation to form a parameter update strategy, and integrate the parameter update strategy into the multi-stage collaborative optimizer (corresponding to steps 3 and 4), so as to perform a model parameter update method as described in any of the above embodiments based on the multi-stage collaborative optimizer (corresponding to step 5).

[0127] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0128] The model parameter updating device provided in an embodiment of the present application is described below. The model parameter updating device described below and the model parameter updating method described above can be referenced to each other.

[0129] like Figure 5 As shown, the present application provides a model parameter updating device 300, the device comprising:

[0130] The model training module 301 is used to iteratively train the target model in a pipeline parallel manner, and the target model is divided into multiple stages for training;

[0131] An optimizer division module 302, for dividing the optimizer state used for parameter updating in each stage into a first optimizer state, a second optimizer state and a dynamic optimizer state based on linear programming;

[0132] A first updating module 303 is used for, in each iteration, when a parameter update is triggered, for each stage, determining the GPU and CPU corresponding to the stage, and loading the dynamic optimizer state of the stage from the CPU to the GPU, so as to perform parameter update in the GPU based on the second optimizer state and the dynamic optimizer state of the stage;

[0133] A second updating module 304 is used to update the parameters corresponding to the first optimizer state of the stage in the CPU based on the first optimizer state of the stage;

[0134] The parameter generation module 305 is used to unload the dynamic optimizer state of this stage to the CPU, and load the updated parameters in the CPU to the GPU to generate the model parameters of this stage.

[0135] In one embodiment, the optimizer partitioning module includes:

[0136] The function construction submodule is used to construct an objective function with the optimization goal of minimizing the parameter update time of the first stage, and determine the corresponding constraint conditions according to the objective function; the objective function uses the first optimizer state, the second optimizer state and the dynamic optimizer state of each stage as decision variables;

[0137] The function solving submodule is used to solve the objective function based on the constraint conditions to determine the optimizer state division situation at each stage when the optimization goal is achieved. The optimizer state division situation includes the division values ​​of the first optimizer state, the second optimizer state and the dynamic optimizer state.

[0138] The optimizer division submodule is used to determine the first optimizer state, the second optimizer state and the dynamic optimizer state in each stage according to the optimizer state division situation of each stage when the optimization target is achieved.

[0139] In one embodiment, the parameter generation module includes:

[0140] A parameter determination submodule, used to determine the updated parameters in the CPU, and after loading the parameters into the GPU, determine the updated first parameter, second parameter and third parameter in the GPU, where the first parameter is the parameter corresponding to the first optimizer state of the stage, the second parameter is the parameter corresponding to the second optimizer state of the stage, and the third parameter is the parameter corresponding to the dynamic optimizer state of the stage;

[0141] The parameter generation submodule is used to perform parameter fusion on the first parameter, the second parameter and the third parameter to obtain the model parameters of this stage.

[0142] In one embodiment, the second update module includes:

[0143] A gradient transmission submodule, used to transmit the first gradient corresponding to the first optimizer state of this stage on the GPU to the CPU;

[0144] The first updating submodule is used to update the parameters corresponding to the first optimizer state of the stage on the CPU based on the first optimizer state of the stage and the first gradient.

[0145] In one embodiment, the first update module includes:

[0146] A gradient determination submodule, used to determine, in the GPU, a second gradient corresponding to the second optimizer state of the stage and a third gradient corresponding to the dynamic optimizer state of the stage;

[0147] A gradient sending submodule, used to receive the gradient sent by the predecessor stage of the stage, and send the second gradient to the successor stage of the stage;

[0148] A gradient generation submodule, for generating a target gradient according to the gradient sent by the predecessor stage of the stage and the second gradient;

[0149] A second updating submodule, used for updating parameters corresponding to the second optimizer state of the stage on the GPU based on the second optimizer state of the stage and the target gradient;

[0150] The third updating submodule is used to update the parameters corresponding to the dynamic optimizer state of this stage on the GPU based on the dynamic optimizer state of this stage and the third gradient.

[0151] In one embodiment, the second updating submodule includes:

[0152] A parameter calculation unit, used to calculate a parameter corresponding to the second optimizer state of the stage according to the second optimizer state of the stage and the target gradient, to obtain a first parameter;

[0153] a parameter receiving unit, configured to receive a parameter sent by a successor stage of the stage, and send a first parameter to a predecessor stage of the stage;

[0154] A parameter updating unit is used to update the parameters corresponding to the second optimizer state of the stage based on the parameters sent by the successor stage of the stage and the first parameters.

[0155] In one of the embodiments, before the step of performing parameter updating in the GPU based on the second optimizer state and the dynamic optimizer state of the stage in the first updating module, the model parameter updating device further includes:

[0156] A gradient determination module, used to determine in the GPU a gradient corresponding to a first optimizer state of the stage, a gradient corresponding to a second optimizer state of the stage, and a gradient corresponding to a dynamic optimizer state of the stage;

[0157] The overflow check module is used to perform an overflow check on the determined gradient to ensure the numerical integrity of the determined gradient.

[0158] The division of each module in the above-mentioned model parameter updating device is only for illustration. In other embodiments, the model parameter updating device can be divided into different modules as needed to complete all or part of the functions of the above-mentioned model parameter updating device. Each module in the above-mentioned model parameter updating device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0159] In one embodiment, the present application also provides a storage medium, which stores computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the model parameter updating method described in any of the above embodiments.

[0160] In one embodiment, the present application also provides a computer device having computer-readable instructions stored therein. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the model parameter updating method described in any one of the above embodiments.

[0161] Indicatively, Figure 6 As shown, Figure 6 This is a schematic diagram of the internal structure of a computer device provided in an embodiment of the present application. The computer device 400 may be provided as a server. Figure 6The computer device 400 includes a processing component 402, which further includes one or more processors, and a memory resource represented by a memory 401 for storing instructions executable by the processing component 402, such as an application. The application stored in the memory 401 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 402 is configured to execute instructions to perform the model parameter updating method of any of the above embodiments.

[0162] The computer device 400 may further include a power supply component 403 configured to perform power management of the computer device 400, a wired or wireless network interface 404 configured to connect the computer device 400 to a network, and an input / output (I / O) interface 405. The computer device 400 may operate based on an operating system stored in the memory 401, such as Windows Server TM, Mac OS X TM, Unix TM, Linux TM, Free BSD TM, or the like.

[0163] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0164] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish an entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the term "include", "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only include those elements, but also include other elements that are not clearly listed, or also include elements inherent to such process, method, article or equipment. In the absence of more restrictions, the elements limited by the sentence "comprise one..." do not exclude the existence of other identical elements in the process, method, article or equipment including the elements. Herein, the singular "one", "one" and "described / the" may also include plural forms, unless the context clearly indicates another way. It should also be understood that the terms "include / comprise" or "have" etc. specify the existence of stated features, wholes, steps, operations, components, parts or combinations thereof, but do not exclude the possibility of the existence or addition of one or more other features, wholes, steps, operations, components, parts or combinations thereof. At the same time, the term "and / or" used in this specification includes any and all combinations of the relevant listed items.

[0165] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can refer to each other.

[0166] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A model parameter updating method, characterized in that: The method comprises: Iteratively training the target model in a pipeline parallel manner, wherein the target model is divided into multiple stages for training; Based on linear programming, the optimizer state used for parameter updating in each stage is divided into a first optimizer state, a second optimizer state and a dynamic optimizer state; In each iteration, when a parameter update is triggered, for each stage, a GPU and a CPU corresponding to the stage are determined, and a dynamic optimizer state of the stage is loaded from the CPU to the GPU, so as to perform a parameter update in the GPU based on the second optimizer state and the dynamic optimizer state of the stage; Based on the first optimizer state of the stage, updating the parameters corresponding to the first optimizer state of the stage in the CPU; The dynamic optimizer state of this stage is unloaded to the CPU, and the parameters updated in the CPU are loaded into the GPU to generate the model parameters of this stage.

2. The model parameter updating method according to claim 1, characterized in that: The method divides the optimizer state for parameter updating in each stage into a first optimizer state, a second optimizer state and a dynamic optimizer state based on linear programming, including: An objective function is constructed with minimizing the parameter update time of the first stage as the optimization goal, and its corresponding constraint conditions are determined according to the objective function; the objective function uses the first optimizer state, the second optimizer state and the dynamic optimizer state of each stage as decision variables; Solving the objective function based on the constraint condition to determine the optimizer state division situation at each stage when the optimization goal is achieved, wherein the optimizer state division situation includes division values ​​of a first optimizer state, a second optimizer state, and a dynamic optimizer state; The first optimizer state, the second optimizer state and the dynamic optimizer state in each stage are determined according to the optimizer state division situation of each stage when the optimization target is achieved.

3. The model parameter updating method according to claim 1, characterized in that: The loading of the updated parameters in the CPU into the GPU to generate the model parameters of this stage includes: After determining the updated parameters in the CPU and loading the parameters into the GPU, determining the updated first parameter, second parameter, and third parameter in the GPU, wherein the first parameter is a parameter corresponding to the first optimizer state of the stage, the second parameter is a parameter corresponding to the second optimizer state of the stage, and the third parameter is a parameter corresponding to the dynamic optimizer state of the stage; The first parameter, the second parameter and the third parameter are fused to obtain the model parameters of this stage.

4. The model parameter updating method according to claim 1, characterized in that: The updating of the parameters corresponding to the first optimizer state of the stage in the CPU based on the first optimizer state of the stage includes: Transmitting a first gradient corresponding to a first optimizer state of the stage on the GPU to the CPU; Parameters corresponding to the first optimizer state of the stage are updated on the CPU based on the first optimizer state of the stage and the first gradient.

5. The model parameter updating method according to claim 1, characterized in that: The performing parameter updating in the GPU based on the second optimizer state and the dynamic optimizer state of the stage includes: Determining, in the GPU, a second gradient corresponding to a second optimizer state of the stage and a third gradient corresponding to a dynamic optimizer state of the stage; receiving a gradient sent by a predecessor stage of the stage, and sending the second gradient to a successor stage of the stage; Generate a target gradient according to the gradient sent by the predecessor stage of the stage and the second gradient; Based on the second optimizer state of the stage and the target gradient, updating the parameters corresponding to the second optimizer state of the stage on the GPU; Based on the dynamic optimizer state of this stage and the third gradient, parameters corresponding to the dynamic optimizer state of this stage are updated on the GPU.

6. The model parameter updating method according to claim 5, characterized in that: The updating of the parameters corresponding to the second optimizer state of the stage on the GPU based on the second optimizer state of the stage and the target gradient includes: Calculate the parameter corresponding to the second optimizer state of this stage according to the second optimizer state of this stage and the target gradient to obtain the first parameter; receiving a parameter sent by a successor stage of the stage, and sending the first parameter to a predecessor stage of the stage; The parameters corresponding to the second optimizer state of this stage are updated based on the parameters sent by the successor stage of this stage and the first parameters.

7. The model parameter updating method according to any one of claims 1 to 6, characterized in that: Before the step of performing parameter updating in the GPU based on the second optimizer state and the dynamic optimizer state of the stage, the method further includes: Determining, in the GPU, a gradient corresponding to a first optimizer state of the stage, a gradient corresponding to a second optimizer state of the stage, and a gradient corresponding to a dynamic optimizer state of the stage; The determined gradient is overflow checked to ensure the numerical integrity of the determined gradient.

8. A model parameter updating device, characterized in that: The device comprises: A model training module, used to iteratively train the target model in a pipeline parallel manner, wherein the target model is divided into multiple stages for training; An optimizer division module, used for dividing the optimizer state used for parameter updating in each stage into a first optimizer state, a second optimizer state and a dynamic optimizer state based on linear programming; A first update module is used for, in each iteration, when a parameter update is triggered, for each stage, determining a GPU and a CPU corresponding to the stage, and loading a dynamic optimizer state of the stage from the CPU to the GPU, so as to perform parameter update in the GPU based on a second optimizer state and a dynamic optimizer state of the stage; A second updating module, configured to update, in the CPU, parameters corresponding to the first optimizer state of the stage based on the first optimizer state of the stage; The parameter generation module is used to unload the dynamic optimizer state of this stage to the CPU, and load the updated parameters in the CPU into the GPU to generate the model parameters of this stage.

9. A storage medium, characterized in that: The storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the model parameter updating method according to any one of claims 1 to 7.

10. A computer device, characterized in that: include: one or more processors, and memory; The memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, the steps of the model parameter updating method according to any one of claims 1 to 7 are performed.

Images