Model parameter update methods, devices, storage media, and computer equipment

Abstract

The model parameter update method, apparatus, storage medium, and computer equipment provided in this application include: iteratively training the target model using a pipelined parallel approach, and dividing the optimizer state in each stage into a first optimizer state, a second optimizer state, and a dynamic optimizer state based on linear programming. In each iteration, when a parameter update is triggered, for each stage, the dynamic optimizer state of that stage is loaded onto the GPU to perform the parameter update, the dynamic optimizer state of that stage is unloaded onto the CPU, and the updated parameters from the CPU are loaded onto the GPU to generate the model parameters for that stage. By setting the dynamic optimizer state to be dynamically loaded between the GPU and the CPU, the dynamic optimizer state can be loaded onto the GPU to perform the parameter update when the GPU is idle, improving the GPU utilization during the parameter update stage and thus improving the training efficiency of the model.

Description

Technical Field

[0001] This application relates to the field of deep learning technology, and in particular to a method, apparatus, storage medium and computer device for updating model parameters. Background Technology

[0002] As the parameters of large language models grow from billions to trillions, traditional single-device training methods are limited by computing resources, resulting in a continuous decrease in training efficiency. To accelerate the model training process, parallel strategies can be used to distribute tasks across multiple GPUs for model training. However, memory limitations become the main obstacle to further scaling up large language models.

[0003] When training large language models on a single or multi-GPU node, the optimizer state is the primary contributor to memory usage. A common approach to reduce optimizer memory consumption is optimizer offloading, which transfers all optimizer state and corresponding update tasks to the CPU. This allows the CPU's computational and memory resources to support large model training, significantly reducing GPU memory usage. However, due to the limited computing power of the CPU, using optimizer offloading for training large language models can lead to reduced model training efficiency. Summary of the Invention

[0004] The purpose of this application is to address at least one of the aforementioned technical shortcomings, particularly the prior art's practice of offloading all or part of the optimizer state and corresponding update tasks to the CPU. This allows the use of CPU computing and memory resources to support large model training, thereby significantly reducing GPU memory usage. However, due to the limited computing power of the CPU, using optimizer offloading for training large language models results in reduced model training efficiency.

[0005] Firstly, this application provides a model parameter update method, the method comprising:

[0006] The target model is trained iteratively using a pipelined parallel approach, and the target model is divided into multiple stages for training.

[0007] Based on linear programming, the optimizer state used for parameter updates in each stage is divided into the first optimizer state, the second optimizer state, and the dynamic optimizer state.

[0008] In each iteration, when a parameter update is triggered, for each stage, the corresponding GPU and CPU are determined, and the dynamic optimizer state of that stage is loaded from the CPU to the GPU, so that the parameter update is performed in the GPU based on the second optimizer state and the dynamic optimizer state of that stage.

[0009] Based on the first optimizer state of this stage, the parameters corresponding to the first optimizer state of this stage are updated in the CPU.

[0010] The dynamic optimizer state for this stage is unloaded to the CPU, and the updated parameters from the CPU are loaded into the GPU to generate the model parameters for this stage.

[0011] In one embodiment, the linear programming-based optimization process divides the optimizer state used for parameter updates in each stage into a first optimizer state, a second optimizer state, and a dynamic optimizer state, including:

[0012] An objective function is constructed with the goal of minimizing the parameter update time in the first stage, and the corresponding constraints are determined based on the objective function. The objective function uses the first optimizer state, the second optimizer state, and the dynamic optimizer state in each stage as decision variables.

[0013] The objective function is solved based on the constraints to determine the optimizer state division at each stage when the optimization objective is achieved. The optimizer state division includes the division values ​​of the first optimizer state, the second optimizer state, and the dynamic optimizer state.

[0014] The first optimizer state, second optimizer state, and dynamic optimizer state in each stage are determined based on the optimizer state division at each stage when the optimization objective is achieved.

[0015] In one embodiment, loading the updated parameters from the CPU into the GPU to generate the model parameters for this stage includes:

[0016] After determining the updated parameters in the CPU and loading them into the GPU, the updated first parameter, second parameter, and third parameter are determined in the GPU. The first parameter is the parameter corresponding to the first optimizer state of this stage, the second parameter is the parameter corresponding to the second optimizer state of this stage, and the third parameter is the parameter corresponding to the dynamic optimizer state of this stage.

[0017] The first parameter, the second parameter, and the third parameter are fused to obtain the model parameters for this stage.

[0018] In one embodiment, updating the parameters corresponding to the first optimizer state of that stage in the CPU based on the first optimizer state of that stage includes:

[0019] The first gradient corresponding to the first optimizer state of this stage on the GPU is transferred to the CPU;

[0020] Based on the first optimizer state and the first gradient of this stage, the parameters corresponding to the first optimizer state of this stage are updated on the CPU.

[0021] In one embodiment, performing parameter updates in the GPU based on the second optimizer state and the dynamic optimizer state of that stage includes:

[0022] In the GPU, a second gradient corresponding to the second optimizer state of this stage and a third gradient corresponding to the dynamic optimizer state of this stage are determined.

[0023] Receive the gradient sent by the preceding stage of this stage, and send the second gradient to the succeeding stage of this stage;

[0024] The target gradient is generated based on the gradient sent by the preceding stage and the second gradient.

[0025] Based on the second optimizer state of this stage and the target gradient, the parameters corresponding to the second optimizer state of this stage are updated on the GPU.

[0026] Based on the dynamic optimizer state of this stage and the third gradient, the parameters corresponding to the dynamic optimizer state of this stage are updated on the GPU.

[0027] In one embodiment, updating the parameters corresponding to the second optimizer state of that stage on the GPU based on the second optimizer state of that stage and the target gradient includes:

[0028] The parameters corresponding to the second optimizer state of this stage are calculated based on the second optimizer state of this stage and the target gradient to obtain the first parameter;

[0029] Receive parameters sent by the subsequent stage of this stage, and send the first parameter to the preceding stage of this stage;

[0030] The parameters corresponding to the second optimizer state of this stage are updated based on the parameters sent by the subsequent stages of this stage and the first parameters.

[0031] In one embodiment, prior to the step of performing parameter updates in the GPU based on the second optimizer state and the dynamic optimizer state of that stage, the method further includes:

[0032] In the GPU, the gradient corresponding to the first optimizer state of the stage, the gradient corresponding to the second optimizer state of the stage, and the gradient corresponding to the dynamic optimizer state of the stage are determined.

[0033] An overflow check is performed on the determined gradient to ensure the numerical integrity of the determined gradient.

[0034] Secondly, this application provides a model parameter updating device, the device comprising:

[0035] The model training module is used to iteratively train the target model in a pipelined parallel manner, wherein the target model is divided into multiple stages for training.

[0036] The optimizer partitioning module is used to divide the optimizer state used for parameter updates in each stage into the first optimizer state, the second optimizer state, and the dynamic optimizer state based on linear programming.

[0037] The first update module is used to determine the GPU and CPU corresponding to each stage when parameter update is triggered in each iteration, and load the dynamic optimizer state of the stage from the CPU to the GPU, so as to perform parameter update in the GPU based on the second optimizer state and dynamic optimizer state of the stage.

[0038] The second update module is used to update the parameters corresponding to the first optimizer state of the stage in the CPU based on the first optimizer state of the stage.

[0039] The parameter generation module is used to unload the dynamic optimizer state of this stage to the CPU and load the updated parameters in the CPU into the GPU to generate the model parameters for this stage.

[0040] Thirdly, this application provides a storage medium storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the model parameter update method as described in any of the above embodiments.

[0041] Fourthly, this application provides a computer device, including: one or more processors, and a memory;

[0042] The memory stores computer-readable instructions, and when the one or more processors execute the computer-readable instructions, they perform the steps of the model parameter update method as described in any of the above embodiments.

[0043] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0044] The model parameter update method, apparatus, storage medium, and computer device provided in this application include: iteratively training a target model in a pipelined parallel manner, dividing the target model into multiple stages for training; then, based on linear programming, dividing the optimizer state used for parameter update in each stage into a first optimizer state, a second optimizer state, and a dynamic optimizer state; in each iteration, when parameter update is triggered, for each stage, determining the corresponding GPU and CPU, and then loading the dynamic optimizer state of that stage from the CPU to the GPU, so that parameter update is performed in the GPU based on the second optimizer state and the dynamic optimizer state of that stage; then, updating the parameters corresponding to the first optimizer state of that stage in the CPU based on the first optimizer state of that stage; finally, unloading the dynamic optimizer state of that stage from the CPU and loading the updated parameters from the CPU to the GPU to generate the model parameters for that stage. By dividing the optimizer state, the dynamic optimizer state is set to be dynamically loaded between the GPU and the CPU. This allows the dynamic optimizer state to be loaded onto the GPU when the GPU is idle to perform parameter updates. This avoids the GPU having a lot of idle time during the parameter update phase due to unloading the optimizer to the CPU, thus improving the utilization of the GPU during the parameter update phase. This saves GPU memory and also improves the training efficiency of the model. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 A flowchart illustrating a model parameter update method provided in an embodiment of this application;

[0047] Figure 2 Comparison example diagrams of model parameter updates provided in embodiments of this application;

[0048] Figure 3 A flowchart illustrating how the optimizer state used for parameter updates in each stage is divided into a first optimizer state, a second optimizer state, and a dynamic optimizer state based on linear programming, as provided in the embodiments of this application.

[0049] Figure 4 An example diagram illustrating the application environment of the model parameter update method provided in this application embodiment;

[0050] Figure 5This is a schematic diagram of the structure of a model parameter updating device provided in an embodiment of this application;

[0051] Figure 6 This is an internal structural diagram of a computer device provided in an embodiment of this application. Detailed Implementation

[0052] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0053] In one embodiment, this application provides a model parameter update method, which is described below using an example of applying the method to a server. It is understood that the model parameter update method can be performed on a single server or a server cluster consisting of multiple servers; this application does not impose any specific limitations on this.

[0054] like Figure 1 As shown, this application provides a model parameter update method, the method comprising:

[0055] S101: The target model is trained iteratively using a pipelined parallel approach.

[0056] The target model is trained in multiple stages.

[0057] In this step, when users have a need to train large language models, due to the large data scale and parameter magnitude of large language models, a pipelined parallel approach can be used to iteratively train the target model. Specifically, this pipelined parallel training method divides the target model into multiple stages along the layer dimension, with each stage assigned to a different computing device (such as a GPU). During forward propagation, data is computed starting from the first stage, and the output is passed to the next stage, and so on, until the last stage obtains the model output. During backpropagation, gradient information is also passed backward according to the stage order. Each stage calculates the gradient of its responsible part and then updates the model parameters it is responsible for based on the gradient. This parallel approach can make full use of computing resources, effectively reduce the overall training time of the model, and exhibit good scalability and efficiency when handling large-scale models and large amounts of data.

[0058] S102: Based on linear programming, the optimizer state used for parameter updates in each stage is divided into the first optimizer state, the second optimizer state, and the dynamic optimizer state.

[0059] In this context, optimizer state refers to the dynamic information maintained by the optimizer during model training to effectively update model parameters. The first optimizer state refers to the optimizer state located on the CPU, the second optimizer state refers to the optimizer state located on the GPU, and the dynamic optimizer state refers to the optimizer state dynamically loaded between the GPU and CPU.

[0060] In this step, a function model for the optimizer partitioning problem can be constructed based on the idea of ​​linear programming. Then, under certain conditions, the optimal solution of this function model is solved to determine the first optimizer state, second optimizer state, and dynamic optimizer state for each stage. These determined first optimizer states, second optimizer states, and dynamic optimizer states for each stage are then placed on the CPU, GPU, and CPU, respectively. It is important to note that the second optimizer state on the GPU for each stage includes itself and the optimizer states of multiple preceding stages.

[0061] It is understandable that by dividing the optimizer state used for parameter updates, the parameter update task can be allocated. Furthermore, since a portion of the optimizer state is divided into dynamic optimizer states, a dynamic adjustment can be made to balance GPU memory and parameter update efficiency. This saves GPU memory to increase the upper limit of model size while saving training time in terms of parameter updates, thereby improving the training efficiency of large models.

[0062] S103: In each iteration, when a parameter update is triggered, for each stage, determine the GPU and CPU corresponding to that stage, and load the dynamic optimizer state of that stage from the CPU to the GPU so that the parameter update can be performed in the GPU based on the second optimizer state and the dynamic optimizer state of that stage.

[0063] Specifically, the condition for triggering parameter updates can be set to when backpropagation is completed in each stage.

[0064] In each iteration, parameter updates are triggered when backpropagation is completed in each stage. For each stage, the GPU and CPU of the device corresponding to that stage are determined. Then, the dynamic optimizer state of that stage is loaded from the CPU to the GPU. This allows a portion of the parameter update task to be performed on the GPU based on the second optimizer state and the dynamic optimizer state of that stage.

[0065] Understandably, loading the dynamic optimizer state onto the GPU during the parameter update phase reduces GPU idle time. Once the parameter update is complete, the dynamic optimizer state is unloaded from the GPU and loaded onto the CPU, thus reducing memory consumption during forward and backward propagation. Therefore, when a parameter update is triggered, the dynamic optimizer state needs to be loaded from the CPU onto the GPU. This allows for the dynamic loading of a portion of the optimizer state (dynamic optimizer state) onto the GPU during GPU idle periods to perform parameter updates, and then unloading it back to the CPU after the update is complete. This fully utilizes the GPU during the parameter update phase and conserves memory.

[0066] S104: Based on the first optimizer state of this stage, update the parameters corresponding to the first optimizer state of this stage in the CPU.

[0067] In this step, the parameters corresponding to the first optimizer state of this stage are updated using the first optimizer state located on the CPU to obtain the updated parameters in the CPU.

[0068] S105: Unload the dynamic optimizer state of this stage to the CPU and load the updated parameters from the CPU to the GPU to generate the model parameters for this stage.

[0069] Once the parameters are updated, the dynamic optimizer state for this stage can be unloaded back onto the CPU, and the updated parameters from the CPU can be loaded onto the GPU to generate the corresponding model parameters for this stage.

[0070] In one example, such as Figure 2 As shown, Figure 2 A comparative example diagram showing model parameter updates provided in embodiments of this application. Figure 2 (a) is an example diagram of the parameter update stage in an existing pipelined parallel training, i.e., individual parameter updates. Figure 2 (b) is an example diagram of the model parameter update method provided in this application, namely, collaborative parameter update. Specifically, in a pipeline containing S stages, due to data dependencies, the i-th stage needs to perform forward propagation of si micro-batches during the warm-up stage, accumulating a total of si activation values. Therefore, the memory usage of each stage is uneven. Figure 2 (a) For example, phase 0 requires 3 forward propagations, while phase 2 only requires 1 forward propagation.

[0071] exist Figure 2In (a), because the optimizer state is offloaded to the CPU to save GPU memory, most parameter update tasks are completed on the CPU, resulting in varying lengths of idle time on the GPU during the parameter update phase (the Bubble area and the white area in the diagram). This leads to low resource utilization and low parameter update efficiency during the parameter update phase. Figure 2 In (b), since the dynamic optimizer state of the corresponding pre-divided stage is loaded from the CPU to the CPU, the GPU can also perform part of the parameter update task in the parameter update stage, thereby reducing the time consumed by parameter update as a whole, improving the resource utilization of the parameter update stage, thereby improving the efficiency of model parameter update and speeding up the model training process.

[0072] The model parameter update method, apparatus, storage medium, and computer device provided in this application include: iteratively training a target model in a pipelined parallel manner, dividing the target model into multiple stages for training; then, based on linear programming, dividing the optimizer state used for parameter update in each stage into a first optimizer state, a second optimizer state, and a dynamic optimizer state; in each iteration, when parameter update is triggered, for each stage, determining the corresponding GPU and CPU, and then loading the dynamic optimizer state of that stage from the CPU to the GPU, so that parameter update is performed in the GPU based on the second optimizer state and the dynamic optimizer state of that stage; then, updating the parameters corresponding to the first optimizer state of that stage in the CPU based on the first optimizer state of that stage; finally, unloading the dynamic optimizer state of that stage from the CPU and loading the updated parameters from the CPU to the GPU to generate the model parameters for that stage. By dividing the optimizer state, the dynamic optimizer state is set to be dynamically loaded between the GPU and the CPU. This allows the dynamic optimizer state to be loaded onto the GPU when the GPU is idle to perform parameter updates. This avoids the GPU having a lot of idle time during the parameter update phase due to unloading the optimizer to the CPU, thus improving the utilization of the GPU during the parameter update phase. This saves GPU memory and also improves the training efficiency of the model.

[0073] like Figure 3 As shown, in one embodiment, the optimizer state used for parameter updates in each stage is divided into a first optimizer state, a second optimizer state, and a dynamic optimizer state based on linear programming, including:

[0074] S201: Construct an objective function with the goal of minimizing the parameter update time of the first stage, and determine its corresponding constraints based on the objective function.

[0075] The objective function uses the states of the first optimizer, the second optimizer, and the dynamic optimizer at each stage as decision variables. Constraints include the first constraint, the second constraint, memory constraints, computation time constraints, and communication time constraints.

[0076] In this step, since the parameter update time of all stages except the first stage is limited by the computation time constraint, we only need to consider the parameter update time of the bottleneck stage (the first stage) to minimize its update time and accelerate the overall parameter update efficiency. Based on this, we can construct an objective function with minimizing the parameter update time of the first stage as the optimization objective, and then determine its corresponding constraints.

[0077] In one example, the constructed objective function and its corresponding constraints can be represented as:

[0078]

[0079]

[0080] In the formula, Minimize means minimizing. This represents the objective function, indicating the first stage on the CPU. The parameter update time, where st represents the constraint condition. , These represent the first constraint and the second constraint, respectively. Representation phase The first optimizer state, Representation phase The state of the dynamic optimizer. Representation phase and stages The second optimizer state, when Time indicates stage or stage The second optimizer state, This represents the total number of optimizer states in the target model. Representation phase Memory constraints, Representation phase The highest memory usage during the training iteration process (including forward propagation, back propagation, and parameter update phases). Representation phase The highest memory usage during forward and backward propagation. Representation phase Peak video memory usage during parameter update phase This indicates the GPU's rated memory. , For preset coefficients, Representation phase computation time constraints, Indicates the stage on the CPU The parameter update time, This represents the forward propagation time of a micro-batch. Indicates communication channel Communication time constraints, Representation phase The corresponding channel group loading time of the dynamic optimizer state. Indicates communication channel The number of communication packets, Indicates communication channel Total loading capacity This represents the backpropagation time of a microbatch. This indicates the available loading time. Among them, and The calculation expression can be represented as:

[0081]

[0082]

[0083] In the formula, Representation phase The memory usage of the model parameters. Representation phase The memory usage of a micro-batch.

[0084] Specifically, Used to account for other video memory overhead, its value ranges from 0 to 1. To account for the relaxation after CPU resources are released, and in the optimization problem corresponding to the objective function, the decision variables can be expressed as follows: , ,in, and for dimensional vector, for 3D matrix. Considering that the computing power of GPUs is significantly higher than that of CPUs, but their video memory is limited and the communication time between GPUs and CPUs is relatively long, GPU video memory constraints, CPU computing time constraints, and communication time constraints are introduced.

[0085] (1) GPU memory constraints: During pipelined parallel training, the peak GPU memory consumption at each stage may occur in two places: one is during the pipeline warm-up stage, i.e., before the backpropagation of the first microbatch is completed, and the other is before global synchronization. To prevent insufficient GPU memory from causing training failure, the GPU memory consumption of these two peaks will be constrained, i.e. and .

[0086] (2) Computation time constraint (CPU computation time constraint): In order to ensure that the parameter update time of each stage conforms to the time of its forward propagation (i.e., the arithmetic sequence), constraints can be imposed on the CPU computation time.

[0087] (3) Communication time constraint: In order to ensure that the dynamic optimizer states loaded by all GPUs linked on the same PCIe channel do not exceed the channel’s transport capacity, a constraint can be imposed on the communication time between the GPU and the CPU, so that the loading time is less than or equal to the available loading time.

[0088] S202: Solve the objective function based on the constraints to determine the optimizer state partitioning at each stage when the optimization objective is achieved.

[0089] The optimizer state division includes the division values ​​of the first optimizer state, the second optimizer state, and the dynamic optimizer state.

[0090] In this step, appropriate solution methods can be used to find the optimal solution of the objective function under constraints, in order to determine the optimizer state partitioning at each stage when the optimization objective is achieved. Specifically, the solution method can be a heuristic algorithm, or it can be the branch and bound method, the cutting plane method, etc., and this application does not impose any specific restrictions on it.

[0091] S203: Determine the first optimizer state, second optimizer state, and dynamic optimizer state in each stage based on the optimizer state division at each stage when the optimization objective is achieved.

[0092] Once the first optimizer state, second optimizer state, and dynamic optimizer state for each stage are determined, the first optimizer state for each stage can be placed on the CPU of the corresponding stage, the second optimizer state for each stage can be placed on the GPU of the corresponding stage, and the dynamic optimizer state for each stage can be placed on the CPU of the corresponding stage.

[0093] In this embodiment, due to the imbalance of GPU memory in each stage during pipeline training, the GPU usage in the parameter update stage, which combines pipeline parallelism and optimizer unloading, is analyzed to determine the optimization objective and construct the objective function. Finally, the optimal solution that satisfies the optimization objective is obtained. The optimizer state partitioning of each stage is configured in advance to maximize GPU utilization, accelerate the model training process, and improve model training efficiency.

[0094] In one embodiment, the updated parameters in the CPU are loaded into the GPU to generate the model parameters for this stage, including:

[0095] S1: Determine the updated parameters in the CPU, load the updated parameters into the GPU, and then determine the updated first, second, and third parameters in the GPU.

[0096] The first parameter is the parameter corresponding to the first optimizer state of this stage, that is, the updated parameter determined in the CPU; the second parameter is the parameter corresponding to the second optimizer state of this stage; and the third parameter is the parameter corresponding to the dynamic optimizer state of this stage.

[0097] In this step, after the parameter update is completed, the updated parameters in the CPU need to be sent to the GPU. At this time, the GPU contains the parameters (second parameters) corresponding to the second optimizer state of this stage after the update is completed, and the parameters (third parameters) corresponding to the dynamic optimizer state of this stage. Then, by combining the updated parameters (first parameters) in the CPU, the updated first, second, and third parameters can be determined and obtained.

[0098] It should be noted that in this application, the transmission between the GPU and the CPU can use PCIe full-duplex transmission.

[0099] S2: Perform parameter fusion on the first, second, and third parameters to obtain the model parameters for this stage.

[0100] Specifically, when the first parameter, the second parameter, and the third parameter are aggregated on the GPU, parameter fusion is performed on the first parameter, the second parameter, and the third parameter is determined as the model parameter for this stage.

[0101] Furthermore, the parameter fusion process for the first, second, and third parameters can employ methods such as weighted average method or incremental update fusion, and this application does not impose specific restrictions on this.

[0102] In this embodiment, after the parameter update is completed, the dynamic optimizer state is unloaded from the GPU and loaded onto the CPU, thus saving GPU memory. Simultaneously, the updated parameters from the CPU are transferred to the GPU to determine the first, second, and third parameters, thereby generating the model parameters for this stage.

[0103] In one embodiment, updating the parameters corresponding to the first optimizer state of that stage in the CPU based on the first optimizer state of that stage includes:

[0104] S1: Transfer the first gradient on the GPU corresponding to the first optimizer state of this stage to the CPU.

[0105] Specifically, since both forward and backward propagation are performed on the GPU at each stage, the gradients obtained at the end of backward propagation at each stage are also stored on the GPU. Since the first optimizer state has been pre-defined on the CPU, the first gradient corresponding to the first optimizer state at that stage on the GPU needs to be transferred to the CPU so that the parameters can be updated on the CPU based on the first gradient and the first optimizer state.

[0106] S2: Based on the first optimizer state and the first gradient of this stage, update the parameters corresponding to the first optimizer state of this stage on the CPU.

[0107] In this embodiment, the first gradient corresponding to the first optimizer state is transmitted to the CPU to enable parameter updates on the CPU based on the first gradient and the first optimizer state.

[0108] In one embodiment, parameter updates are performed in the GPU based on the second optimizer state and the dynamic optimizer state of this stage, including:

[0109] S1: Determine the second gradient corresponding to the second optimizer state of this stage and the third gradient corresponding to the dynamic optimizer state of this stage in the GPU.

[0110] S2: Receives the gradient sent by the predecessor stage of this stage, and sends the second gradient to the successor stage of this stage.

[0111] The preceding stage refers to the stage before this stage, and the successor stage refers to the stage after this stage.

[0112] In this step, the gradient received from the predecessor stage can be combined with the gradient calculated by itself during backpropagation to obtain a complete gradient for updating the model parameters responsible for this stage. Similarly, the second gradient is sent to the successor stage so that the successor stage can also perform similar complete gradient calculation and parameter update.

[0113] S3: Generate the target gradient based on the gradient sent by the preceding stage and the second gradient.

[0114] S4: Based on the second optimizer state and the target gradient of this stage, update the parameters corresponding to the second optimizer state of this stage on the GPU.

[0115] S5: Based on the dynamic optimizer state of this stage and the third gradient, update the parameters corresponding to the dynamic optimizer state of this stage on the GPU.

[0116] In this embodiment, gradient transfer is used to update the parameters corresponding to the second optimizer state of this stage in the GPU. Since the dynamic optimizer state is loaded on the GPU during the parameter update stage, the update of the parameters corresponding to the dynamic optimizer state of this stage is also performed in the GPU.

[0117] In one embodiment, the parameters corresponding to the second optimizer state of that stage are updated on the GPU based on the second optimizer state and the target gradient of that stage, including:

[0118] S1: Calculate the parameters corresponding to the second optimizer state of this stage based on the second optimizer state and the objective gradient, and obtain the first parameter.

[0119] S2: Receive the parameters sent by the successor stage of this stage, and send the first parameter to the predecessor stage of this stage.

[0120] S3: Update the parameters corresponding to the second optimizer state of this stage based on the parameters sent by the subsequent stage and the first parameter.

[0121] In this embodiment, since updating the parameters corresponding to the second optimizer state involves the gradient of the predecessor stage, the first parameter is sent to the predecessor stage of this stage. Since the second gradient corresponding to the second optimizer state of this stage is sent to the successor stage, the parameter sent by the successor stage of this stage is received, thereby completing the update of the parameters corresponding to the second optimizer state of this stage.

[0122] In one embodiment, prior to the step of performing parameter updates in the GPU based on the second optimizer state and the dynamic optimizer state of this stage, the model parameter update method further includes:

[0123] S1: Determine the gradient corresponding to the first optimizer state of this stage, the gradient corresponding to the second optimizer state of this stage, and the gradient corresponding to the dynamic optimizer state of this stage in the GPU.

[0124] S2: Perform an overflow check on the determined gradient to ensure the numerical integrity of the determined gradient.

[0125] In this embodiment, firstly, from the perspective of model training stability, checking can prevent the training process from spiraling out of control due to excessively large or small gradient values. When gradient overflow occurs, it may cause abnormal parameter updates, such as causing model parameters to become infinite or non-numeric (NaN). Overflow checking can detect this potential crisis in time, ensuring that the training process can smoothly move towards optimization. Secondly, regarding model accuracy, appropriate gradient values ​​allow the model to adjust parameters based on reasonable gradient information, which helps improve the accuracy and speed of model convergence.

[0126] In one example, such as Figure 4 As shown, Figure 4 This diagram illustrates an application environment example of the model parameter update method provided in this embodiment. Figure 4 In this process, the user provides a dataset, a model (i.e., the target model in this application), and a training configuration (corresponding to step 1). Then, a preset analyzer is used to analyze the data provided by the user to determine the bubble status, which can reflect the idle time of the GPU. The bubble status is then sent to a preset bubble scheduler (corresponding to step 2). At this time, the bubble scheduler can determine the optimizer state division of each stage according to the bubble status to form a parameter update strategy, and integrate the parameter update strategy into a multi-stage collaborative optimizer (corresponding to steps 3 and 4) to execute the model parameter update method as described in any of the above embodiments based on the multi-stage collaborative optimizer (corresponding to step 5).

[0127] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0128] The model parameter updating apparatus provided in the embodiments of this application is described below. The model parameter updating apparatus described below can be referred to in correspondence with the model parameter updating method described above.

[0129] like Figure 5 As shown, this application provides a model parameter update device 300, the device comprising:

[0130] The model training module 301 is used to iteratively train the target model in a pipelined parallel manner, and the target model is divided into multiple stages for training.

[0131] The optimizer partitioning module 302 is used to partition the optimizer state used for parameter updates in each stage into a first optimizer state, a second optimizer state, and a dynamic optimizer state based on linear programming.

[0132] The first update module 303 is used to determine the GPU and CPU corresponding to each stage when parameter update is triggered in each iteration, and load the dynamic optimizer state of the stage from the CPU to the GPU so as to perform parameter update in the GPU based on the second optimizer state and the dynamic optimizer state of the stage.

[0133] The second update module 304 is used to update the parameters corresponding to the first optimizer state of the stage in the CPU based on the first optimizer state of the stage.

[0134] The parameter generation module 305 is used to unload the dynamic optimizer state of this stage to the CPU and load the updated parameters in the CPU to the GPU to generate the model parameters for this stage.

[0135] In one embodiment, the optimizer partitioning module includes:

[0136] The function construction submodule is used to construct the objective function with the optimization objective of minimizing the parameter update time of the first stage, and to determine the corresponding constraints based on the objective function; the objective function uses the first optimizer state, the second optimizer state, and the dynamic optimizer state of each stage as decision variables;

[0137] The function solving submodule is used to solve the objective function based on the constraints to determine the optimizer state division at each stage when the optimization objective is achieved. The optimizer state division includes the division values ​​of the first optimizer state, the second optimizer state, and the dynamic optimizer state.

[0138] The optimizer partitioning submodule is used to determine the first optimizer state, the second optimizer state, and the dynamic optimizer state in each stage based on the optimizer state partitioning at each stage when the optimization goal is achieved.

[0139] In one embodiment, the parameter generation module includes:

[0140] The parameter determination submodule is used to determine the updated parameters in the CPU, load the parameters into the GPU, and then determine the updated first, second, and third parameters in the GPU. The first parameter is the parameter corresponding to the first optimizer state of this stage, the second parameter is the parameter corresponding to the second optimizer state of this stage, and the third parameter is the parameter corresponding to the dynamic optimizer state of this stage.

[0141] The parameter generation submodule is used to fuse the first, second, and third parameters to obtain the model parameters for this stage.

[0142] In one embodiment, the second update module includes:

[0143] The gradient transfer submodule is used to transfer the first gradient on the GPU corresponding to the first optimizer state of this stage to the CPU.

[0144] The first update submodule is used to update the parameters corresponding to the first optimizer state of the current stage on the CPU based on the first optimizer state and the first gradient of the current stage.

[0145] In one embodiment, the first update module includes:

[0146] The gradient determination submodule is used to determine the second gradient corresponding to the second optimizer state of this stage and the third gradient corresponding to the dynamic optimizer state of this stage in the GPU.

[0147] The gradient sending submodule is used to receive the gradient sent by the preceding stage of this stage and send the second gradient to the succeeding stage of this stage.

[0148] The gradient generation submodule is used to generate the target gradient based on the gradient sent by the preceding stage and the second gradient of this stage.

[0149] The second update submodule is used to update the parameters corresponding to the second optimizer state of this stage on the GPU based on the second optimizer state and the target gradient of this stage.

[0150] The third update submodule is used to update the parameters corresponding to the dynamic optimizer state of this stage on the GPU based on the dynamic optimizer state of this stage and the third gradient.

[0151] In one embodiment, the second update submodule includes:

[0152] The parameter calculation unit is used to calculate the parameters corresponding to the second optimizer state of the current stage based on the second optimizer state and the objective gradient of the current stage, so as to obtain the first parameter.

[0153] The parameter receiving unit is used to receive the parameters sent by the subsequent stage of this stage and send the first parameter to the preceding stage of this stage.

[0154] The parameter update unit is used to update the parameters corresponding to the second optimizer state of this stage based on the parameters sent by the subsequent stage and the first parameter.

[0155] In one embodiment, prior to the step in the first update module of performing parameter updates in the GPU based on the second optimizer state and the dynamic optimizer state of that stage, the model parameter update apparatus further includes:

[0156] The gradient determination module is used to determine the gradient corresponding to the first optimizer state of the current stage, the gradient corresponding to the second optimizer state of the current stage, and the gradient corresponding to the dynamic optimizer state of the current stage in the GPU.

[0157] The overflow check module is used to perform overflow checks on the determined gradient to ensure the numerical integrity of the determined gradient.

[0158] The division of modules in the above-described model parameter update device is merely illustrative. In other embodiments, the model parameter update device can be divided into different modules as needed to complete all or part of its functions. Each module in the above-described model parameter update device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0159] In one embodiment, this application also provides a storage medium storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the model parameter update method as described in any of the above embodiments.

[0160] In one embodiment, this application also provides a computer device storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the model parameter update method as described in any of the above embodiments.

[0161] Indicatively, such as Figure 6 As shown, Figure 6 This is a schematic diagram of the internal structure of a computer device 400 provided in an embodiment of this application. The computer device 400 can be provided as a server. (Refer to...) Figure 6The computer device 400 includes a processing component 402, which further includes one or more processors, and memory resources represented by memory 401 for storing instructions, such as application programs, that can be executed by the processing component 402. The application programs stored in memory 401 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 402 is configured to execute instructions to perform the model parameter update method of any of the above embodiments.

[0162] The computer device 400 may also include a power supply component 403 configured to perform power management of the computer device 400, a wired or wireless network interface 404 configured to connect the computer device 400 to a network, and an input / output (I / O) interface 405. The computer device 400 may operate on an operating system stored in memory 401, such as Windows Server™, Mac OS X™, Unix™, Linux™, Free BSD™, or similar.

[0163] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0164] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising a…" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. In this document, the singular forms "a," "an," and "the" may also include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms “comprising / including” or “having” specify the presence of the stated features, wholes, steps, operations, components, parts or combinations thereof, but do not exclude the possibility of the presence or addition of one or more other features, wholes, steps, operations, components, parts or combinations thereof. Meanwhile, the term “and / or” as used in this specification includes any and all combinations of the associated listed items.

[0165] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

[0166] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for updating model parameters, characterized in that, The method includes: The target model is trained iteratively using a pipelined parallel approach, and the target model is divided into multiple stages for training. An objective function is constructed with the goal of minimizing the parameter update time of the first stage, and the corresponding constraints are determined based on the objective function. The objective function uses the first optimizer state, the second optimizer state, and the dynamic optimizer state of each stage as decision variables. The first optimizer state refers to the optimizer state located on the CPU, the second optimizer state refers to the optimizer state located on the GPU, and the dynamic optimizer state refers to the optimizer state dynamically loaded between the CPU and the GPU. The objective function is solved based on the constraints to determine the optimizer state division at each stage when the optimization objective is achieved. The optimizer state division includes the division values ​​of the first optimizer state, the second optimizer state, and the dynamic optimizer state. The first optimizer state, second optimizer state, and dynamic optimizer state in each stage are determined based on the optimizer state division at each stage when the optimization objective is achieved. In each iteration, when a parameter update is triggered, for each stage, the corresponding GPU and CPU are determined, and the dynamic optimizer state of that stage is loaded from the CPU to the GPU, so that the parameter update is performed in the GPU based on the second optimizer state and the dynamic optimizer state of that stage. Based on the first optimizer state of this stage, the parameters corresponding to the first optimizer state of this stage are updated in the CPU. The dynamic optimizer state for this stage is unloaded to the CPU, and the updated parameters from the CPU are loaded into the GPU to generate the model parameters for this stage.

2. The model parameter update method according to claim 1, characterized in that, The process of loading the updated parameters from the CPU into the GPU to generate the model parameters for this stage includes: After determining the updated parameters in the CPU and loading them into the GPU, the updated first parameter, second parameter, and third parameter are determined in the GPU. The first parameter is the parameter corresponding to the first optimizer state of this stage, the second parameter is the parameter corresponding to the second optimizer state of this stage, and the third parameter is the parameter corresponding to the dynamic optimizer state of this stage. The first parameter, the second parameter, and the third parameter are fused to obtain the model parameters for this stage.

3. The model parameter update method according to claim 1, characterized in that, The process of updating the parameters corresponding to the first optimizer state of this stage in the CPU based on the first optimizer state of this stage includes: The first gradient corresponding to the first optimizer state of this stage on the GPU is transferred to the CPU; Based on the first optimizer state and the first gradient of this stage, the parameters corresponding to the first optimizer state of this stage are updated on the CPU.

4. The model parameter update method according to claim 1, characterized in that, The parameter update performed in the GPU based on the second optimizer state and the dynamic optimizer state at this stage includes: In the GPU, a second gradient corresponding to the second optimizer state of this stage and a third gradient corresponding to the dynamic optimizer state of this stage are determined. Receive the gradient sent by the preceding stage of this stage, and send the second gradient to the succeeding stage of this stage; The target gradient is generated based on the gradient sent by the preceding stage and the second gradient. Based on the second optimizer state of this stage and the target gradient, the parameters corresponding to the second optimizer state of this stage are updated on the GPU. Based on the dynamic optimizer state of this stage and the third gradient, the parameters corresponding to the dynamic optimizer state of this stage are updated on the GPU.

5. The model parameter update method according to claim 4, characterized in that, The step of updating the parameters corresponding to the second optimizer state of this stage on the GPU based on the second optimizer state of this stage and the target gradient includes: The parameters corresponding to the second optimizer state of this stage are calculated based on the second optimizer state of this stage and the target gradient to obtain the first parameter; Receive parameters sent by the subsequent stage of this stage, and send the first parameter to the preceding stage of this stage; The parameters corresponding to the second optimizer state of this stage are updated based on the parameters sent by the subsequent stages of this stage and the first parameters.

6. The model parameter update method according to any one of claims 1 to 5, characterized in that, Prior to the step of performing parameter updates in the GPU based on the second optimizer state and the dynamic optimizer state of this stage, the method further includes: In the GPU, the gradient corresponding to the first optimizer state of the stage, the gradient corresponding to the second optimizer state of the stage, and the gradient corresponding to the dynamic optimizer state of the stage are determined. An overflow check is performed on the determined gradient to ensure the numerical integrity of the determined gradient.

7. A model parameter update device, characterized in that, The device includes: The model training module is used to iteratively train the target model in a pipelined parallel manner, wherein the target model is divided into multiple stages for training. An optimizer partitioning module is used to construct an objective function with the optimization objective of minimizing the parameter update time of the first stage, and to determine the corresponding constraints based on the objective function. The objective function uses the first optimizer state, the second optimizer state, and the dynamic optimizer state of each stage as decision variables. The first optimizer state refers to the optimizer state located on the CPU, the second optimizer state refers to the optimizer state located on the GPU, and the dynamic optimizer state refers to the optimizer state dynamically loaded between the CPU and the GPU. The objective function is solved based on the constraints to determine the optimizer state partitioning of each stage when the optimization objective is achieved. The optimizer state partitioning includes the partition values ​​of the first optimizer state, the second optimizer state, and the dynamic optimizer state. The first optimizer state, the second optimizer state, and the dynamic optimizer state in each stage are determined according to the optimizer state partitioning of each stage when the optimization objective is achieved. The first update module is used to determine the GPU and CPU corresponding to each stage when parameter update is triggered in each iteration, and load the dynamic optimizer state of the stage from the CPU to the GPU, so as to perform parameter update in the GPU based on the second optimizer state and dynamic optimizer state of the stage. The second update module is used to update the parameters corresponding to the first optimizer state of the stage in the CPU based on the first optimizer state of the stage. The parameter generation module is used to unload the dynamic optimizer state of this stage to the CPU and load the updated parameters in the CPU into the GPU to generate the model parameters for this stage.

8. A storage medium, characterized in that: The storage medium stores computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the model parameter update method as described in any one of claims 1 to 6.

9. A computer device, characterized in that, include: One or more processors, and memory; The memory stores computer-readable instructions that, when executed by the one or more processors, perform the steps of the model parameter update method as described in any one of claims 1 to 6.

Images