A distributed learning information compression method based on shared random projection

Abstract

The application discloses a distributed learning information compression method based on shared random projection and belongs to the technical field of machine learning and neural network optimization. The application uses a Gaussian random vector shared between distributed system devices to project and compress gradient information and reconstruct the gradient information. The gradient vector is compressed to an arbitrary low-dimensional space before communication, and then the gradient vector is reconstructed after communication. The application can be directly deployed in any existing first-order optimizer, communication architecture and training framework, can ensure the unbiasedness of the reconstructed vector and the boundedness of the variance, and thus can maintain high quality of training in distributed learning. Compared with existing gradient compression methods, the application has stronger universality, higher training speed, better training effect and simpler deployment mode.

Description

Technical Field

[0001] This invention belongs to the fields of machine learning, artificial intelligence, distributed training, and neural network optimization technology, and specifically relates to a distributed learning information compression method based on shared random projection. Background Technology

[0002] In recent years, deep learning (DL) technology has made significant progress in many fields, including image, text, and video processing, and has been successfully applied in numerous areas, including computer vision (e.g., image classification, object detection) and natural language processing (e.g., language modeling, machine translation). The key to the performance improvement of models trained using deep learning technology on most tasks lies in the substantial increase in the amount of training data and model size. Correspondingly, as the model size increases, the memory and computational overhead of training large models has exceeded the capabilities of a single computing device (such as a GPU). Therefore, in recent years, distributed training of large models on large cluster devices (such as GPUs and TPUs) has become the mainstream technology in academia and industry for researching large models. The distribution and communication of models and data across multiple devices has become a key factor affecting the training efficiency of large models. A major strategy for distributed training is 3D parallel technology, which integrates data parallelism, pipeline parallelism, and tensor parallelism to improve the efficiency of distributed training on GPU clusters.

[0003] The problem with 3D parallel training is that as the number of model parameters increases, the communication cost required to synchronize data across multiple GPUs becomes increasingly expensive. Therefore, efficient communication training strategies are crucial for improving the efficiency, scalability, and applicability of large model training. A common technique for reducing communication costs is gradient compression, which compresses the original gradient vectors during communication, significantly improving the efficiency of vector communication between GPUs. However, gradient compression has some drawbacks; potential information loss during compression can degrade model training quality. Therefore, to balance training efficiency and quality, research on gradient compression methods often focuses on two key issues: how to ensure training performance after using compression techniques, and how to design methods suitable for modern communication architectures. Existing commonly used information compression methods often struggle to address both of these issues. Low-rank quantization methods are often based on master-server communication architectures, which are incompatible with current advanced ring-based and tree-based communication architectures; low-rank methods introduce additional computational overhead, and their compatibility with the default Adam-type optimizer in deep learning is uncertain. Furthermore, integrating these methods into existing full-precision optimizers typically requires significant modifications and additional work to ensure compatibility and maintain performance.

[0004] Currently, achieving efficient training of large models remains a significant challenge. A key issue urgently needs to be addressed: does a simple information compression method exist that can maintain high training quality and significant communication efficiency in distributed learning, independent of the optimizer and communication framework? Summary of the Invention

[0005] To overcome the shortcomings of existing technologies and address the problems in efficient training of large models, this invention proposes a distributed learning information compression method based on shared random projection, named SEPARATE (Simple and gEneral Plug-in for grAdient compRession in modern 1A-scale Training process). SEPARATE can serve as a simple and universal plug-in compatible with common gradient-based optimizers (such as SGD and Adam) in various training frameworks. During GPT-2-Medium model training, it achieves the same training accuracy as the full-precision Adam optimizer in only half the training time.

[0006] The technical solution provided by this invention is as follows:

[0007] A distributed learning information compression method based on shared random projection includes the following steps:

[0008] 1) Preprocess the text data to obtain the training text encoding;

[0009] 2) Construct a large-scale language model trained on text data and initialize the parameters of the model to be trained:

[0010] 3) Model the training task as an optimization problem, select a gradient-based first-order optimizer to solve the optimization problem, and perform the following steps during the optimizer's solution process:

[0011] 3-1) Using the selected optimizer, perform forward and backward propagation on the model parameters to obtain the gradient information corresponding to the model parameters on each device n in the distributed cluster. Where g represents gradient information, the subscript n indicates that the gradient is local information obtained on the nth device, and k indicates that the current iteration is the kth round.

[0012] 3-2) The gradient information obtained in this round Cumulative error stored on the current device Add them together to get Where e represents the cumulative error value stored on the device, the subscript n indicates that the gradient is the local information obtained on the nth device, and k indicates that the current iteration is the kth round.

[0013] 3-3) Generate m independent and identically distributed Gaussian random vectors ξ1,…,ξ m ~N(0,I d ), and calculate Projection in the directions of these random vectors [p 1,n ,…,p m,n ],in <·,·> represent the inner product operation between two vectors, and [·] represents concatenating the results of these inner product operations into a new vector. N(0,I) d ) represents a d-dimensional standard normal distribution;

[0014] 3-4) For different devices n, [p] 1,n ,…,p m,n Perform global communication to obtain

[0015] 3-5) On each device n The information is reconstructed as follows Use The gradient vector estimate is used to update the optimizer's state and model parameters;

[0016] 4) After a certain number of iterations, save the model and test and evaluate the trained model.

[0017] Furthermore, in step 2), if the model is pre-trained from scratch, the model parameters are initialized to zero; if the model is fine-tuned, the pre-trained model parameters are read from the specified path.

[0018] Furthermore, in step 4), if the model is pre-trained from scratch, the function value of its loss function after the iteration is completed is recorded; if the model is fine-tuned, the saved model is evaluated on the selected downstream task and the evaluation accuracy is recorded.

[0019] Compared with the prior art, the beneficial effects of the present invention include:

[0020] The core objective of this invention is to design a general information compression method that allows gradient vectors to be compressed to an arbitrary low-dimensional space before communication and then reconstructed after communication. The proposed SEPARATE method can be directly deployed on any existing first-order optimizer, communication architecture, and training framework; that is, it can be considered a plug-in that can be easily implemented on top of an existing optimizer with just a few lines of code. This invention combines the landscape characteristics of real-world tasks, using Gaussian random vectors shared among devices in a distributed system to project, compress, and reconstruct gradient information. This invention ensures the unbiasedness and bounded variance of the reconstructed vectors, thus maintaining high-quality training in distributed learning. Furthermore, users can choose the compression ratio according to their device settings; the compression ratio can exceed 32 (the typical gradient quantization algorithm's compression ratio is at most 32). The SEPARATE method can achieve high-quality, efficient communication for large language model pre-training and fine-tuning tasks, improving the training effect of distributed learning without affecting training efficiency. Therefore, the specific technical advantages of this invention include:

[0021] (I) This invention proposes a novel gradient compression method called SEPARATE, which can be applied to existing gradient-based optimizers and training frameworks with just a few lines of code. Utilizing a novel error feedback technique, SEPARATE maintains the same training performance as the full-precision optimizer both theoretically and experimentally. Furthermore, users can freely choose the compression ratio according to their device settings, even exceeding 32. Compared to existing gradient compression methods, this invention offers greater versatility, faster training speed, better training results, and a simpler deployment method.

[0022] (ii) The large model pre-training task based on the present invention can achieve training accuracy comparable to that of the full-precision optimizer in a shorter time. Deploying the present invention on Adam can reduce the training time of large models to half that of training with the full-precision optimizer.

[0023] (III) The large model fine-tuning task based on this invention has achieved higher evaluation accuracy than the full-precision optimizer on many downstream tasks. The distributed learning information compression algorithm proposed in this invention can improve the performance of the model after fine-tuning while reducing communication overhead by 16 to 32 times. Attached Figure Description

[0024] Figure 1 This is a flowchart of the information compression method based on shared random projection proposed in this invention. Detailed Implementation

[0025] The technical details of the present invention are further described below with reference to examples, but this does not limit the scope of the present invention in any way.

[0026] This invention provides an application flow for SEPARATE, and gives a detailed description of the implementation steps of SEPARATE in practical applications.

[0027] This invention constructs an information compression method, named SEPARATE, applicable to all gradient-based first-order optimizers and communication frameworks. SEPARATE can be easily deployed as a plugin for large-scale real-world model training tasks. In practical applications, the technical details of SEPARATE deployment are described below. The specific implementation includes the following steps:

[0028] 1) Preprocess the text data to obtain the training text encoding;

[0029] 2) Construct a large-scale language model trained on text data and initialize the parameters of the model to be trained:

[0030] 3) Approximate the training task as an optimization problem, select a gradient-based first-order optimizer (usually Adam) to solve the optimization problem, and perform the following steps during the optimizer's solution process:

[0031] 3-1) Using the selected optimizer, perform forward and backward propagation on the model parameters to obtain the gradient information corresponding to the model parameters on each device n in the distributed cluster. Where g represents gradient information, the subscript n indicates that the gradient is local information obtained on the nth device, and k indicates that the current iteration is the kth round.

[0032] 3-2) The gradient information obtained in this round Cumulative error stored on the current device Add them together to get Where e represents the cumulative error value stored on the device, the subscript n indicates that the gradient is the local information obtained on the nth device, and k indicates that the current iteration is the kth round.

[0033] 3-3) Generate m independent and identically distributed Gaussian random vectors ξ1,…,ξ m ~N(0,I d ), and calculate Projection in the directions of these random vectors [p 1,n ,…,p m,n ],in <·,·> represent the inner product operation between two vectors, and [·] represents concatenating the results of these inner product operations into a new vector. N(0,I) d ) represents a d-dimensional standard normal distribution;

[0034] 3-4) For different devices n, [p] 1,n ,…,pm,n Perform global communication to obtain The global communication method is based on the communication architecture adopted by the distributed cluster, which is generally a ring-based or tree-based communication architecture. The linear property of random projection ensures that this method can be deployed under any communication architecture.

[0035] 3-5) On each device n The information is reconstructed as follows Use The gradient vector is used as an estimate to update the state of the optimizer and the model parameters. The method of parameter update is determined by the selected optimizer. The method proposed in this invention compresses the gradient vector, so it can be deployed on any gradient-based first-order optimizer (such as SGD and Adam).

[0036] 3-6) Calculate the moving average of the compression error and cumulative error under the current round as the new cumulative error and store it as follows:

[0037]

[0038] in This represents the cumulative error stored on device n in the (k+1)th iteration, where β is a parameter of the moving average, which can be adjusted as needed. After a certain number of iterations, the cumulative error is reset to zero.

[0039] 4) After a certain number of iterations, save the model and test and evaluate the trained model.

[0040] SEPARATE projects local gradient information from different devices using a shared random vector, but requires only a small amount of additional computational cost. This is because the computational process of projecting using m independent and identically distributed Gaussian random vectors can be expressed as multiplying the original gradient vector by a d×m Gaussian random matrix. This form is well supported in modern computing architectures (such as CUDA), and the additional computational overhead in practical applications is negligible compared to the forward and backward propagation overhead in deep learning training.

[0041] This invention further designs a computational parallelization mechanism to achieve further acceleration in practical applications. Through the hook mechanism in PyTorch, this invention designs a parallel implementation of compressed computation and backpropagation computation. The hook mechanism adds a "hook" to the model's backpropagation process. Before the optimizer calculates the gradient vector of the previous layer after obtaining the gradient vector of the next layer, the hook mechanism initiates and performs the compression and communication of the gradient of the next layer. Simultaneously, the optimizer performs the computation of the gradient vector of the previous layer in parallel. Therefore, by rationally arranging the order of computation and communication, the time cost of gradient compression can be hidden in the backpropagation computation process, thus achieving the same computational speed as the full-precision optimizer. Since SEPARATE greatly reduces communication overhead, the training speed of the model (which can be considered as the sum of computational and communication overhead) is significantly improved.

[0042] The SEPARATE method naturally possesses a degree of versatility. SEPARATE can be applied to ring-based or tree-based communication architectures, which typically rely on reduce-scatter operations and have limited support for quantization methods. Unlike quantization methods, SEPARATE is unaffected by numerical anomalies such as overflow or underflow, thus providing excellent support for reduce-scatter operations. SEPARATE also supports the partitioning strategy commonly used when training large language models; even if gradients are split across different nodes, the linear nature of random projection ensures the correctness of the SEPARATE results.

[0043] This invention optimizes the memory overhead of SEPARATE. Due to error feedback and the storage of random vectors, SEPARATE introduces additional memory overhead for training. This invention sets up a random variable buffer, generating a portion of Gaussian random vectors before training begins and storing them in this buffer for the initial training phase. After use, the random vectors in the buffer are updated and overwritten. By reusing this portion of memory to store the random vector stream, memory costs are reduced.

[0044] This invention utilizes the pre-training of large models, employing Adam to train large models (such as GPT-2-Medium) on specified datasets and deploying the SEPARATE method to obtain pre-trained models with specified accuracy more efficiently.

[0045] This invention utilizes the SEPARATE method to train large models and achieves faster training speeds. Specific technical details are as follows:

[0046] This invention trained the GPT-2-Medium model from scratch on an 8x NVIDIA 3090 24G GPU cluster. It used the OpenWebtext dataset, running a total of 300,000 iterations and processing 50 billion tokens. The PyTorch DDP training framework was used, without model or pipeline parallelization. Before training, the model parameters were set as follows: global batch size of 8x512, learning rate of 6.0e-4, and a cosine annealing learning rate decay strategy. During training, the first 2000 iterations were warm-started, and the learning rate was decayed to a minimum of 6.0e-5 in the subsequent 298,000 iterations. Gradient accumulation was used, with a gradient accumulation step of 4. Global gradient norm clipping was employed, set to 1, and the two parameters of the Adam optimizer were set to 0.9 and 0.95.

[0047] This invention trains a GPT-2-Medium model using both a standard Adam optimizer and a SEPARATE-based Adam optimizer. During backpropagation, the invention utilizes the SEPARATE method to compress and communicate the obtained layer gradients. Specifically, the invention reads a random vector of a given size from a random vector buffer, reassembles it into a d×m dimensional random matrix, and multiplies it by the result of adding the original gradient vector and the accumulated error vector stored on the device. This results in a compressed low-dimensional vector for communication, and the accumulated error and random vector buffer are updated after communication. For the SEPARATE method, different compression ratios can be set, i.e., the factor by which m is reduced compared to D. For quantization methods, this factor has an upper bound of 32, while the SEPARATE method can be freely set according to requirements. This invention tests the training effect of SEPARATE under different factors, comparing the convergence speed of iteration steps and clock time, as shown in Table 1. All methods use the same training settings.

[0048] Table 1: Convergence speed of this invention compared to training GPT-2-Medium with the ordinary Adam optimizer

[0049]

[0050] Table 1 shows that the SEPARATE method of this invention can accelerate the training speed of GPT-2-Medium pre-training by 2 times by balancing the compression ratio and single-step iteration cost. Therefore, the SEPARATE method can maintain training performance while improving communication and training efficiency. Furthermore, this invention also tested the single-step time cost of ordinary Adam and SEPARATE with different compression ratios on GPT-2-Small and GPT-2-Meduim. The effect of SEPARATE becomes more significant with increasing model size, thus SEPARATE is more effective when training large models. Higher compression ratios can still reduce single-step time costs, but may lead to more information loss; therefore, setting an appropriate compression ratio is a trade-off in practice.

[0051] This invention is used to fine-tune large models. The large model (such as LLaMA2-7B) is fine-tuned on a specified dataset using Adam and the SEPARATE method is deployed. The resulting model is then evaluated on a specified downstream task to obtain a fine-tuned model with better performance.

[0052] This invention utilizes the SEPARATE method to fine-tune large models and evaluates them on a large number of downstream tasks. The technical details are as follows:

[0053] The algorithm implementation steps are largely the same as those for training GPT-2-Medium. For the experimental setup, this invention fine-tuned the LLaMA2-7B model on the alpaca-gpt4 dataset within the PyTorch FSDP communication framework, and compared the proposed compression method based on shared random projection with common quantization and low-rank methods. This invention fine-tuned LLaMA2-7B for three epochs on an 8x NVIDIA A6000 48G GPU cluster. To adapt to the device, the data parallelism was set to 4, the model parallelism to 2, pipelined parallelism was not used, the global batch size was set to 32, the learning rate to 2e-5, the gradient clipping to 2, and the gradient accumulation steps to 8. This invention selected diverse benchmarks to evaluate the fine-tuned model, including commonsense reasoning, world knowledge, mathematical computation, and code generation. The results are shown in Table 2. All methods used the same settings.

[0054] Table 2 Evaluation results of the present invention and existing compression methods for fine-tuning LLaMA2-7B on multiple benchmarks.

[0055] Adam 49.39 15.69 19.40 3.07 46.09 72.31 53.90 57.07 26.45 38.15 PowerSGD 50.78 18.65 22.40 2.52 40.57 62.43 42.71 52.88 30.12 35.90 1-bit Adam 62.08 16.53 16.80 5.15 49.57 49.21 37.97 52.88 22.04 34.69 ZeRO++ 57.49 18.42 22.20 1.72 49.88 43.03 30.85 47.61 33.97 33.91 SEPARATE 57.18 20.17 21.40 4.18 49.25 72.66 49.83 52.99 29.73 39.72

[0056] The results in Table 2 show that this invention outperforms other low-rank or quantized optimizers with error feedback and the ZeRO++ optimizer in all downstream tasks, and even surpasses the performance of the uncompressed baseline Adam optimizer. Other gradient compression methods lag far behind SEPARATE and the baseline Adam, especially in commonsense reasoning tasks. This is because their error feedback based on current iteration cannot handle the accumulated errors throughout the process, while this invention uses a moving average of historical errors to stabilize the error and an error reset mechanism to eliminate useless information, thus exhibiting a significant advantage.

[0057] It should be noted that the purpose of disclosing the embodiments is to help further understand the present invention. However, those skilled in the art will understand that various substitutions and modifications are possible without departing from the scope of the present invention and the appended claims. Therefore, the present invention should not be limited to the content disclosed in the embodiments, and the scope of protection of the present invention is defined by the scope of the claims.

Claims

1. A distributed learning information compression method based on shared random projection, characterized in that, Includes the following steps: 1) Preprocess the text data to obtain the training text encoding; 2) Construct a large-scale language model trained on text data and initialize the parameters of the model to be trained: 3) Model the training task as an optimization problem, select a gradient-based first-order optimizer to solve the optimization problem, and perform the following steps during the optimizer's solution process: 3-1) Using the selected optimizer, perform forward and backward propagation on the model parameters to obtain the gradient information corresponding to the model parameters on each device n in the distributed cluster. Where g represents gradient information, the subscript n indicates that the gradient is local information obtained on the nth device, and k indicates that the current iteration is the kth round. 3-2) The gradient information obtained in this round Cumulative error stored on the current device Add them together to get Where e represents the cumulative error value stored on the device, the subscript n indicates that the gradient is the local information obtained on the nth device, and k indicates that the current iteration is the kth round. 3-3) Generate m independent and identically distributed Gaussian random vectors ξ1,…,ξ m ~N(0,I d ), and calculate Projection in the directions of these random vectors [p 1,n ,…,p m,n ],in <·,·> represent the inner product operation between two vectors, and [·] represents concatenating the results of these inner product operations into a new vector. N(0,I) d ) represents a d-dimensional standard normal distribution; 3-4) For different devices n, [p] 1,n ,…,p m,n Perform global communication to obtain 3-5) On each device n The information is reconstructed as follows Use The gradient vector estimate is used to update the optimizer's state and model parameters; 4) After a certain number of iterations, save the model and test and evaluate the trained model.

2. The distributed learning information compression method based on shared random projection as described in claim 1, characterized in that, In step 3), the gradient-based first-order optimizer is either SGD or Adam.

3. The distributed learning information compression method based on shared random projection as described in claim 1, characterized in that, In steps 3-4), the global communication method is based on the communication architecture adopted by the distributed cluster, which is either a ring-based communication architecture or a tree-based communication architecture.

4. The distributed learning information compression method based on shared random projection as described in claim 1, characterized in that, In step 3), the moving average of the compression error and cumulative error in the current round is calculated as the new cumulative error and stored as follows: in This represents the cumulative error stored on device n in the (k+1)th iteration, where β is the parameter of the moving average. After a certain number of iterations, the cumulative error is reset to zero.

5. The distributed learning information compression method based on shared random projection as described in claim 1, characterized in that, In step 2), if the model is pre-trained from scratch, the model parameters are initialized to zero; if the model is fine-tuned, the pre-trained model parameters are read from the specified path.

6. The distributed learning information compression method based on shared random projection as described in claim 5, characterized in that, Set up a random variable buffer, generate some Gaussian random vectors before training begins and store them in the buffer for the initial training phase, and update and overwrite the random vectors in the buffer after use.

7. The distributed learning information compression method based on shared random projection as described in claim 1, characterized in that, If the model is pre-trained from scratch in step 4), the function value of its loss function after the iteration is completed is recorded; if the model is fine-tuned, the saved model is evaluated on the selected downstream task and the evaluation accuracy is recorded.

Images