Distributed training method and device for large language model, equipment and medium
By designing a hybrid gradient update mechanism and a method for quantizing gradient residuals, the problem of high communication volume in distributed training of large language models is solved, thereby improving computational efficiency and training speed.
Patent Information
- Application Number
- CN202411172902.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-26
- Publication Date
- 2026-03-13
AI Technical Summary
Existing distributed training methods for large language models suffer from high communication volume and low computational efficiency during gradient and parameter transmission, failing to effectively utilize the local gradient processing capabilities of worker nodes.
A hybrid gradient update mechanism is adopted, including simple gradient update mode and complex gradient update mode. The working nodes process the local gradient and transmit it to the central node. The communication volume is reduced by quantizing the gradient residual. The central node calculates the global gradient and adjusts the model parameters.
By reducing the amount of communication between worker nodes and the central node, the computational efficiency and training speed of the model are improved.
Smart Images
Figure CN121660026A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of large language model technology, and specifically relates to a distributed training method, apparatus, device and medium for large language models. Background Technology
[0002] In the field of large language models, distributed machine learning methods are typically used to train models using gradient descent in order to quickly obtain the optimal parameters of the model. In distributed machine learning, the gradient descent algorithm can be parallelized, thereby improving computational efficiency. Specifically, distributed gradient descent divides the dataset into multiple small blocks, each node calculates the gradient in parallel, then the gradients of all nodes are aggregated, and finally the parameters are updated. This parallel computing method can significantly reduce computation time and improve the running efficiency of the algorithm.
[0003] Existing distributed training methods for large language models typically use parameter servers, Ring All Reduce, or All Reduce to transmit gradients and parameters between worker nodes and the central node. However, when transmitting gradients, worker nodes calculate local gradients and then transmit them all to the central node for aggregation. The worker nodes do not process the local gradients, resulting in technical problems such as large communication volume and low model computation efficiency. Summary of the Invention
[0004] In view of the above analysis, the embodiments of the present invention aim to provide a distributed training method, apparatus, device and medium for large language models to solve one or more of the above-mentioned problems existing in the prior art.
[0005] The objective of this invention is achieved as follows:
[0006] A first aspect of this invention provides a distributed training method for a large language model, comprising:
[0007] Obtain the training dataset;
[0008] The training dataset is input into a large language model for training, including:
[0009] The training dataset is divided into multiple subsets;
[0010] Each working node randomly selects a subset, calculates the local gradient of its respective loss function, processes the local gradient through a hybrid gradient update mechanism, and transmits the processing result to the central node.
[0011] The average value of the processing results calculated by the central node is used as the global gradient. The parameters of the large language model are adjusted according to the global gradient. The adjusted parameters of the model are broadcast to each working node for the next iteration until the iteration converges.
[0012] The hybrid gradient update mechanism includes a simple gradient update mode and a complex gradient update model. When the current iteration is in the simple gradient update mode, the working node directly transmits the local gradient to the central node. When the current iteration is in the complex gradient update mode, the working node calculates the mean square error between the local gradient and the global gradient in at least the past two iterations, calculates the gradient residual when the mean square error is minimized, and transmits the gradient residual to the central node.
[0013] Further, the calculation of the gradient residual when the mean squared error is minimized includes: the working node calculates the difference between the local gradient and the global gradient of the iteration corresponding to the minimization of the mean squared error, and sets the difference as the gradient residual.
[0014] Further, transmitting the gradient residual to the central node includes: quantizing the gradient residual, and transmitting the quantized gradient residual and the time sequence number of the iteration cycle corresponding to minimizing the mean square error to the central node.
[0015] Further, the quantization of the gradient residual includes:
[0016] Converting the gradient residual from floating-point to fixed-point numbers specifically includes:
[0017] Find each element [v1,...v] in the gradient residual. i ,...v n The absolute value of ] abs([[v1,...v i ,...v n ]]), and extract the maximum absolute value, denoted as a. max ;
[0018] The quantization precision is calculated based on the maximum value and expressed as follows:
[0019] Based on the quantization precision, each element in the gradient residual is converted into a fixed-point number using a constraint function, represented as: clip(round(v i ·p))
[0020] Here, clip(·) represents the restriction function, and round(·) represents the rounding function.
[0021] Furthermore, the central node calculates the average value of the processing result as the global gradient, including: when the current iteration is in a simple gradient update mode, the central node directly calculates the average value of all local gradients and uses the average value as the global gradient; when the current iteration is in a complex gradient update mode, the central node performs inverse recovery based on the quantized gradient residual and time sequence number to obtain the original local gradient and calculates the average value of all original local gradients and uses the average value as the global gradient.
[0022] Further, adjusting the parameters of the large language model based on the global gradient includes:
[0023] The parameters of the large language model in the current iteration, the step size of the gradient descent in the current iteration, and the global gradient are used to adjust the parameters of the large language model as the parameters for the next iteration, as shown below:
[0024]
[0025] Where w(t+1) represents the parameters of the next iteration, w(t) represents the parameters of the large language model in the current iteration, and c(t) represents the step size of the gradient descent in the current iteration. This represents the global gradient.
[0026] Furthermore, it also includes optimizing the parameters of the large language model through an adaptive learning rate optimization algorithm during each iteration.
[0027] A second aspect of the present invention provides a distributed training device for a large language model, comprising:
[0028] The acquisition module is used to acquire the training dataset;
[0029] The training module, used to input the training dataset into a large language model for training, includes:
[0030] The training dataset is divided into multiple subsets;
[0031] Each working node randomly selects a subset, calculates the local gradient of its respective loss function, processes the local gradient through a hybrid gradient update mechanism, and transmits the processing result to the central node.
[0032] The average value of the processing results calculated by the central node is used as the global gradient. The parameters of the large language model are adjusted according to the global gradient. The adjusted parameters of the model are broadcast to each working node for the next iteration until the iteration converges.
[0033] The hybrid gradient update mechanism includes a simple gradient update mode and a complex gradient update model. When the current iteration is in the simple gradient update mode, the working node directly transmits the local gradient to the central node. When the current iteration is in the complex gradient update mode, the working node calculates the mean square error between the local gradient and the global gradient in at least the past two iterations, calculates the gradient residual when the mean square error is minimized, and transmits the gradient residual to the central node.
[0034] A third aspect of the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the computer program, when executed by the processor, implements the distributed training method for a large language model as described in any embodiment.
[0035] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the distributed training method for a large language model as described in any embodiment.
[0036] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:
[0037] The distributed training method for large language models provided by this invention designs a hybrid gradient update mechanism, including simple and complex gradient update modes, to process the local gradients calculated by worker nodes. The processed results are then transmitted to the central node, thereby reducing the communication volume between worker nodes and the central node, and thus improving the computational efficiency of the model. Attached Figure Description
[0038] To more clearly illustrate the technical solutions in the embodiments of this specification or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings.
[0039] Figure 1 A flowchart of the distributed training method for a large language model provided in Embodiment 1 of the present invention;
[0040] Figure 2 This is a schematic diagram of a distributed training device for a large language model provided in Embodiment 2 of the present invention;
[0041] Figure 3 This is a schematic diagram of the electronic device architecture provided in Embodiment 3 of the present invention. Detailed Implementation
[0042] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. It should be noted that, unless otherwise specified, the implementation methods and features in the implementation methods in this disclosure can be combined, separated, interchanged, and / or rearranged. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0043] Example 1
[0044] A specific embodiment of the present invention, such as Figure 1 As shown, a distributed training method for a large language model is disclosed, including:
[0045] S1. Obtain the training dataset;
[0046] S2. Input the training dataset into the large language model for training, specifically including:
[0047] S201. Divide the training dataset into multiple subsets;
[0048] S202. Each working node randomly selects a subset, calculates the local gradient of its respective loss function, processes the local gradient through a hybrid gradient update mechanism, and transmits the processing result to the central node.
[0049] S203. The average value of the processing result calculated by the central node is used as the global gradient. The parameters of the large language model are adjusted according to the global gradient. The adjusted parameters of the model are broadcast to each working node for the next iteration until the iteration converges.
[0050] The hybrid gradient update mechanism includes a simple gradient update mode and a complex gradient update model. When the current iteration is in the simple gradient update mode, the working node directly transmits the local gradient to the central node. When the current iteration is in the complex gradient update mode, the working node calculates the mean square error between the local gradient and the global gradient in at least the past two iterations, calculates the gradient residual when the mean square error is minimized, and transmits the gradient residual to the central node.
[0051] For example, the ratio between the simple gradient update mode and the complex gradient update mode in the hybrid gradient update mechanism can be agreed upon before training, every ten iterations.
[0052] Compared with existing technologies, the distributed training method for large language models provided in this embodiment reduces the amount of communication between worker nodes and the central node by designing a hybrid gradient update mechanism that includes simple gradient update mode and complex gradient update mode, thereby improving the computational efficiency of the model.
[0053] In this embodiment, calculating the gradient residual when minimizing the mean square error includes:
[0054] The working node calculates the difference between the local gradient and the global gradient of the iteration corresponding to the minimization of the mean square error, and sets the difference as the gradient residual.
[0055] Specifically, after each worker node calculates its local gradient, it needs to process the local gradient through a hybrid gradient update mechanism and then transmit the processing result to the central node. When the iteration is in a complex gradient update mode, the worker node first calculates the mean square error between the local gradient calculated in the current iteration and the global gradient in at least the previous two iterations, then finds the minimum mean square error, and then calculates the difference between the local gradient in the current iteration and the global gradient in the iteration with the minimum mean square error, and uses this difference as the gradient residual.
[0056] In this embodiment, transmitting the gradient residual to the central node includes:
[0057] The gradient residual is quantized, and the quantized gradient residual and the time sequence number of the iteration cycle corresponding to the minimization of the mean square error are transmitted to the central node.
[0058] Specifically, after the working nodes calculate the gradient residuals, they need to be further processed to reduce the communication volume during transmission between the working nodes and the central node. The specific processing measures are as follows:
[0059] The working node quantizes the gradient residual, including:
[0060] Converting the gradient residual from floating-point to fixed-point numbers specifically includes:
[0061] Find each element [v1,...v] in the gradient residual. i ,...v n The absolute value of ] abs([[v1,...v i ,...v n ]]), and extract the maximum absolute value, denoted as a. max ;
[0062] The quantization precision is calculated based on the maximum value and expressed as follows:
[0063] Based on the quantization precision, each element in the gradient residual is converted into a fixed-point number using a constraint function, represented as: clip(round(v i ·p))
[0064] Here, clip(·) represents the restriction function, and round(·) represents the rounding function.
[0065] In this embodiment, the average value of the processing result calculated by the central node is used as the global gradient, including:
[0066] When the current iteration is in simple gradient update mode, the central node directly calculates the average of all local gradients and uses the average as the global gradient.
[0067] When the current iteration is in a complex gradient update mode, the central node performs inverse recovery based on the quantized gradient residual and time sequence number to obtain the original local gradient and calculate the average value of all original local gradients, and uses the average value as the global gradient.
[0068] Specifically, after receiving the processing results of the local gradient from the worker nodes, the central node performs further processing based on the current iteration state. If the current iteration is in simple gradient update mode, the average value of the processing results is directly used as the global gradient. If the current iteration is in complex gradient update mode, since the received processing results are quantized, the quantized gradient residuals need to be processed to restore the original local gradient of the worker nodes in reverse order for the accuracy of the results, and then the average value is calculated as the global gradient.
[0069] In this embodiment, adjusting the parameters of the large language model based on the global gradient includes:
[0070] The parameters of the large language model in the current iteration, the step size of the gradient descent in the current iteration, and the global gradient are used to adjust the parameters of the large language model as the parameters for the next iteration, as shown below:
[0071]
[0072] Where w(t+1) represents the parameters of the next iteration, w(t) represents the parameters of the large language model in the current iteration, and c(t) represents the step size of the gradient descent in the current iteration. This represents the global gradient.
[0073] In some embodiments, it also includes:
[0074] In each iteration, the parameters of the large language model are optimized using an adaptive learning rate optimization algorithm.
[0075] For example, the adaptive learning rate optimization algorithm can be Adam, RMSprop, or AdaGrad, and the specific optimization steps are as follows:
[0076] (I) Initialize model parameters and hyperparameters of the adaptive learning rate optimization algorithm;
[0077] (II) During the calculation of local gradients at the working node, the learning rate of the parameters is updated using the adaptive learning rate optimization algorithm.
[0078] (III) Adjust the gradient of the parameters according to the updated learning rate and calculate the updated local gradient;
[0079] (IV) The updated local gradient is transmitted to the central node, which calculates the global gradient based on the received local gradient.
[0080] (V) The central node uses an adaptive learning rate optimization algorithm to update the parameters of the large language model based on the global gradient;
[0081] (VI) Broadcast the updated model parameters to each working node for the next iteration.
[0082] Example 2
[0083] This embodiment provides a distributed training device for a large language model, such as... Figure 2 As shown, it includes:
[0084] The acquisition module is used to acquire the training dataset;
[0085] The training module, used to input the training dataset into a large language model for training, includes:
[0086] The training dataset is divided into multiple subsets;
[0087] Each working node randomly selects a subset, calculates the local gradient of its respective loss function, processes the local gradient through a hybrid gradient update mechanism, and transmits the processing result to the central node.
[0088] The average value of the processing results calculated by the central node is used as the global gradient. The parameters of the large language model are adjusted according to the global gradient. The adjusted parameters of the model are broadcast to each working node for the next iteration until the iteration converges.
[0089] The hybrid gradient update mechanism includes a simple gradient update mode and a complex gradient update model. When the current iteration is in the simple gradient update mode, the working node directly transmits the local gradient to the central node. When the current iteration is in the complex gradient update mode, the working node calculates the mean square error between the local gradient and the global gradient in at least the past two iterations, calculates the gradient residual when the mean square error is minimized, and transmits the gradient residual to the central node.
[0090] Example 3
[0091] This embodiment provides an electronic device, such as... Figure 3 As shown, it includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements the distributed training method for the large language model as described in any of the above embodiments.
[0092] Example 4
[0093] This embodiment provides a computer-readable storage medium storing a computer program thereon. When the program is executed by a processor, it implements the distributed training method for a large language model as described in any of the above embodiments.
[0094] Computer-readable storage media include both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.
[0095] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.
[0096] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.
[0097] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A distributed training method for a large language model, characterized in that, include: Obtain the training dataset; The training dataset is input into a large language model for training, including: The training dataset is divided into multiple subsets; Each working node randomly selects a subset, calculates the local gradient of its respective loss function, processes the local gradient through a hybrid gradient update mechanism, and transmits the processing result to the central node. The average value of the processing results calculated by the central node is used as the global gradient. The parameters of the large language model are adjusted according to the global gradient. The adjusted parameters of the model are broadcast to each working node for the next iteration until the iteration converges. The hybrid gradient update mechanism includes a simple gradient update mode and a complex gradient update model. When the current iteration is in the simple gradient update mode, the working node directly transmits the local gradient to the central node. When the current iteration is in the complex gradient update mode, the working node calculates the mean square error between the local gradient and the global gradient in at least the past two iterations, calculates the gradient residual when the mean square error is minimized, and transmits the gradient residual to the central node.
2. The distributed training method for a large language model according to claim 1, characterized in that, The calculation of the gradient residual when minimizing the mean square error includes: The working node calculates the difference between the local gradient and the global gradient of the iteration corresponding to the minimization of the mean square error, and sets the difference as the gradient residual.
3. The distributed training method for a large language model according to claim 1, characterized in that, Transmitting the gradient residual to the central node includes: The gradient residual is quantized, and the quantized gradient residual and the time sequence number of the iteration cycle corresponding to the minimization of the mean square error are transmitted to the central node.
4. The distributed training method for a large language model according to claim 3, characterized in that, The quantization of the gradient residual includes: Converting the gradient residual from floating-point to fixed-point numbers specifically includes: Find each element [v1,...v] in the gradient residual. i ,...v n The absolute value of ] abs([[v1,...v i ,...v n ]]), and extract the maximum absolute value, denoted as a. max ; The quantization precision is calculated based on the maximum value and expressed as follows: Based on the quantization precision, each element in the gradient residual is converted into a fixed-point number using a constraint function, represented as: clip(round(v i ·p)) Here, clip(·) represents the restriction function, and round(·) represents the rounding function.
5. The distributed training method for a large language model according to claim 4, characterized in that, The average value of the processing results calculated by the central node is used as the global gradient, including: When the current iteration is in simple gradient update mode, the central node directly calculates the average of all local gradients and uses the average as the global gradient. When the current iteration is in a complex gradient update mode, the central node performs inverse recovery based on the quantized gradient residual and time sequence number to obtain the original local gradient and calculate the average value of all original local gradients, and uses the average value as the global gradient.
6. The distributed training method for a large language model according to claim 1, characterized in that, The step of adjusting the parameters of the large language model based on the global gradient includes: The parameters of the large language model in the current iteration, the step size of the gradient descent in the current iteration, and the global gradient are used to adjust the parameters of the large language model as the parameters for the next iteration, as shown below: Where w(t+1) represents the parameters of the next iteration, w(t) represents the parameters of the large language model in the current iteration, and c(t) represents the step size of the gradient descent in the current iteration. This represents the global gradient.
7. The distributed training method for a large language model according to claim 6, characterized in that, Also includes: In each iteration, the parameters of the large language model are optimized using an adaptive learning rate optimization algorithm.
8. A distributed training device for a large language model, characterized in that, The device includes: The acquisition module is used to acquire the training dataset; The training module, used to input the training dataset into a large language model for training, includes: The training dataset is divided into multiple subsets; Each working node randomly selects a subset, calculates the local gradient of its respective loss function, processes the local gradient through a hybrid gradient update mechanism, and transmits the processing result to the central node. The average value of the processing results calculated by the central node is used as the global gradient. The parameters of the large language model are adjusted according to the global gradient. The adjusted parameters of the model are broadcast to each working node for the next iteration until the iteration converges. The hybrid gradient update mechanism includes a simple gradient update mode and a complex gradient update model. When the current iteration is in the simple gradient update mode, the working node directly transmits the local gradient to the central node. When the current iteration is in the complex gradient update mode, the working node calculates the mean square error between the local gradient and the global gradient in at least the past two iterations, calculates the gradient residual when the mean square error is minimized, and transmits the gradient residual to the central node.
9. An electronic device, characterized in that, It includes a memory and a processor, the memory storing a computer program that, when executed by the processor, implements the distributed training method for a large language model as described in any one of claims 1-7.
10. A storage medium, characterized in that, It stores a computer program, which, when executed by a processor, implements the distributed training method for the large language model as described in any one of claims 1-7.