LLM distributed training method, system and device and storage medium

Abstract

The invention discloses an LLM distributed training method, system and device and a storage medium. The method comprises the steps of collecting data and performing data cleaning and preprocessing; dividing the preprocessed data into a plurality of subsets; a part of data subsets serve as a training set, the remaining data subsets serve as a verification set, and the data subsets are stored on different computing nodes in a distributed mode; each computing node performs distributed training on the LLM by using the training set, and each computing node outputs a preliminarily trained LLM when the maximum training round is reached by taking minimization of a loss function of the LLM as a target; integrating the preliminarily trained LLMs in the computing nodes to obtain an integrated LLM; evaluating the integrated LLM by using the verification set and the evaluation index, and adjusting training parameters of the LLM to obtain an optimal LLM; and performing automatic content creation by using the optimal LLM. The resource utilization rate and the calculation performance are further improved, the problem of high calculation complexity is solved, and the overall training efficiency is improved.

Description

Technical Field

[0001] The present invention relates to large language model technology, and in particular to a distributed training method, system, device and storage medium of an LLM. Background Art

[0002] The background of Local Overfitting Optimization (LOO) in Large Language Models (LLM) can be traced back to the development of machine learning and deep learning. As the scale of neural networks continues to increase, the complexity of the models and the amount of data have also increased significantly. Large-scale language models, such as GPT-3 and BERT, have achieved remarkable results in natural language processing (NLP) tasks. However, as the scale of models increases and the data sets become more diverse, the problem of overfitting of the models has become more and more obvious.

[0003] Overfitting refers to the phenomenon that a model performs well on the training data but performs poorly on new data. This is because the model memorizes the noise and details of the training data during training instead of learning the true patterns of the data. Overfitting is particularly prominent for large-scale language models because they often need to process a large amount of heterogeneous data from different fields and tasks. In this case, the model may become overly dependent on certain frequently occurring patterns, resulting in excellent performance on specific subsets but unsatisfactory performance on other subsets.

[0004] To address this problem, researchers have proposed local overfitting optimization technology. The basic idea of ​​this technology is to improve the generalization ability of the model by detecting and adjusting the overfitting behavior of the model on a specific data subset. Specifically, the local overfitting optimization technology includes the following steps: First, the training data is sharded and divided into several subsets. Each subset represents a pattern or field in the data. Then, during the training process, the performance of the model on each subset is monitored in real time to identify the overfitting behavior that may occur in the model on certain subsets. This can be achieved by observing the changes in the loss value of the model on the subset.

[0005] Secondly, take corresponding optimization measures for the detected overfitting behavior. You can reduce the risk of overfitting by increasing the regularization term and reducing the complexity of the model. You can also adjust the learning rate to slow down the learning speed of the model on a specific subset to avoid overfitting of the subset. In addition, you can also use data enhancement technology to increase the diversity of data and reduce the model's dependence on specific patterns by expanding and transforming the training data.

[0006] Finally, during the entire training process, the above steps are repeated continuously to dynamically adjust the model parameters to ensure that the model performs evenly on all data subsets. In this way, the local overfitting optimization technique can not only improve the overall performance of the model, but also significantly improve the generalization ability of the model in different tasks and fields.

[0007] The development of local overfitting optimization technology is inseparable from the advancement of large-scale computing resources. With the continuous development of GPU and distributed computing technology, researchers can apply local overfitting optimization technology on larger data sets and more complex models. However, this also brings new challenges, such as high computational cost and complex implementation. Therefore, in practical applications, researchers need to balance the improvement of model performance and the consumption of computing resources and choose the best solution.

[0008] In general, local overfitting optimization technology provides an effective method for training and optimizing large-scale language models. By detecting and adjusting the overfitting behavior of the model on a specific data subset, researchers can significantly improve the generalization ability and overall performance of the model. However, the high computational complexity and implementation difficulty of this technology also require researchers to continuously explore new optimization methods and technical means to obtain the best results in practical applications.

[0009] Problems with existing technologies: high computational complexity. This technology requires real-time monitoring and adjustment of the model's performance on multiple local data subsets, which will greatly increase the consumption of computing resources and training time. Especially for large-scale data sets and complex LLMs, this computational overhead will become more obvious, causing the training process to become very slow. In addition, the real-time parameter adjustment process also requires precise algorithm support and a lot of debugging work, which further increases the difficulty and cost of technical implementation. Therefore, although local overfitting optimization technology has the potential to improve model performance, its high computational cost and complex implementation process have become a major obstacle in practical applications. Summary of the invention

[0010] The present invention aims at the deficiencies in the prior art and provides a distributed training method, system, device and storage medium for LLM, which utilizes multiple computing nodes in a cluster to work together to speed up the model training process.

[0011] To achieve the above object, the present invention adopts the following technical solutions: A distributed training method for LLM includes the following steps: Collect data and perform data cleaning and preprocessing; Divide the preprocessed data into several subsets; use part of the data subsets as training sets and the remaining data subsets as validation sets, and store the data subsets in a distributed manner on different computing nodes; Each computing node uses the training set to perform distributed training on the LLM, with the goal of minimizing the loss function of the LLM. When the maximum number of training rounds is reached, each computing node outputs a preliminarily trained LLM. Each round of distributed training includes the following process: (a) Each computing node reads at least one data subset in the training set; (b) Each computing node independently calculates the data gradient of the data subset; the data gradient is the partial derivative of the loss function of the LLM with respect to the parameters of the LLM; (c) Summarize data gradients; (d) Each computing node synchronously updates the parameters of the LLM based on the data gradient; Integrate the preliminarily trained LLMs in each computing node to obtain an integrated LLM; Use the validation set and evaluation indicators to evaluate the integrated LLM, adjust the training parameters of the LLM, and obtain the best LLM; Leverage the best of LLM for automated content creation.

[0012] To optimize the above technical solutions, the specific measures taken also include: Furthermore, the method of dividing the preprocessed data into several subsets is a cross-validation method; The distributed storage of data subsets on different computing nodes specifically includes: using a distributed file system or a cloud storage service to perform distributed storage of data subsets, assigning a unique identifier to each data subset when storing the data subsets, and uploading it to the distributed storage system; using scripts or automated tools to batch upload and manage data subsets.

[0013] Furthermore, the method of aggregating data gradients is implemented through a central parameter server or a decentralized method; In the central parameter server approach, each compute node sends the calculated gradients to the central server, which aggregates the gradients of all compute nodes and updates the LLM parameters; The decentralized approach is that each node exchanges gradient information with other nodes. The communication mechanism between nodes includes message passing interface and remote procedure call, and finally a consistent gradient summary result is reached among all nodes.

[0014] Furthermore, during the distributed training process, the deep learning framework used is TensorFlow or PyTorch, and mixed precision training is enabled by setting the precision mode of the model and optimizer; If you use TensorFlow, use the mixed precision API to set the model's computation mode to mixed precision. If you use PyTorch, use the automatic mixed precision package to manage the switch between 16-bit floating point and 32-bit floating point computations. In the process of distributed training of LMM, the forward propagation and back propagation calculations will be partially performed using 16-bit floating point numbers; forward propagation refers to the process of input data being calculated through LLM output, and back propagation is the process of adjusting LMM parameters based on the error between the output and the target value; in mixed precision training, most of the forward propagation calculations use 16-bit floating point numbers, while gradient calculations and key accumulation operations use 32-bit floating point numbers; Use dynamic loss scaling to scale the loss function value to within the 16-bit floating point representation range, and then scale it back before updating the LMM parameters. At the end of each training step, the calculated gradients are accumulated into 32-bit floating point variables, and the accumulated gradients are used to update the parameters of the LMM.

[0015] Furthermore, the computing node is a GPU cluster. Before the distributed training LLM, the process of GPU cluster virtualization and device grid management using the Ray framework and Alpa architecture is also included. The specific implementation steps are: First, configure and initialize the GPU cluster and ensure that Ray and Alpa are installed on each computing node. When configuring a GPU cluster with Ray installed, specify the IP address and resource information of each node in the cluster. Start the entire cluster using the GPU cluster startup script with Ray installed and ensure that each node can communicate with each other. Then, define computing tasks and Actor models in the Ray framework. Tasks are the most basic computing units in Ray and can be executed asynchronously on different nodes. Actor is a computing unit with state that can share state between multiple tasks. Computing tasks are distributed to different nodes for execution through Ray. Using Alpa's automatic partitioning and device grid management capabilities, large-scale computing tasks can be broken down into smaller subtasks and assigned to different nodes in the GPU cluster for parallel computing.

[0016] Furthermore, each node of the GPU cluster also has the SMDDP library installed to improve the efficiency of cluster communication during distributed training; In the distributed training process based on the SMDDP library, the tools provided by the SMDDP library are used to enable the nodes of the GPU cluster to process different data subsets in parallel, update the parameters of the LLM synchronously, and adjust the batch size and data transmission path of the data transmission; Use the monitoring tools provided by the SMDDP library to monitor the progress and performance indicators of training in real time, and adjust the training configuration and communication parameters in a timely manner.

[0017] Furthermore, the device grid is the core mechanism of the Alpa framework for managing computing resources. By defining the device grid, the distribution of computing tasks on different devices is specified, and the configuration of the device grid is adjusted according to specific computing needs. Specifically, the progress of tasks and the resource usage of the cluster are tracked through the monitoring tool of the Ray framework, and the task allocation strategy and the configuration of the device grid are adjusted according to the monitoring results.

[0018] The present invention also proposes a distributed training system for LLM, comprising: Data acquisition module, used to collect data and perform data cleaning and preprocessing; The data partitioning and storage module is used to divide the preprocessed data into several subsets; a part of the data subsets are used as training sets, and the remaining data subsets are used as validation sets, and the data subsets are distributedly stored on different computing nodes; GUP cluster, each computing node in the cluster is a GPU. Each computing node uses the training set to perform distributed training on LLM, with the goal of minimizing the loss function of LLM. When the maximum number of training rounds is reached, each computing node outputs a preliminarily trained LLM; each round of distributed training includes the following process: (a) Each computing node reads at least one data subset in the training set; (b) Each computing node independently calculates the data gradient of the data subset; the data gradient is the partial derivative of the loss function of the LLM with respect to the parameters of the LLM; (c) Summarize data gradients; (d) Each computing node synchronously updates the parameters of the LLM based on the data gradient; The model integration module is used to integrate the preliminarily trained LLMs in each computing node to obtain an integrated LLM; The model adjustment module is used to evaluate the integrated LLM using the validation set and the evaluation index, adjust the training parameters of the LLM, and obtain the best LLM; the best LLM is used for automatic content creation.

[0019] The present invention also proposes an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the distributed training method of LLM as described above is implemented.

[0020] The present invention also provides a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the distributed training method of LLM as described above.

[0021] The beneficial effects of the present invention are: 1. The present invention significantly distributes the computational load and speeds up the training by sharding the training data and processing them in parallel on multiple computers or GPU clusters; 2. The present invention adopts a distributed gradient descent algorithm to synchronously update model parameters. The update process of model parameters can be distributed to each node. Each node independently calculates the data gradient of its subset, and then aggregates these gradients and synchronously updates the model parameters. In this way, not only can the calculation time of a single iteration be significantly reduced, but also the existing computing resources can be fully utilized to improve the overall training efficiency. In addition, through GPU cluster virtualization and optimization of collective communication operations, resource utilization and computing performance are further improved, and the problem of high computational complexity is solved.

[0022] 3. Adopting mixed precision training technology, by using 16-bit floating point numbers for calculation, the training process is accelerated and memory usage is reduced without significantly affecting the accuracy of the model. This can not only significantly reduce the calculation time of a single iteration and improve the overall training efficiency, but also make full use of existing computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A flow chart of the distributed training method of LLM of the present invention; Figure 2 This is the network topology diagram of the GUP cluster. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0025] Embodiment 1

[0026] The present invention proposes a distributed training method for LLM, the flow chart of which is as follows: Figure 1 As shown, the following steps are included: Collect data and perform data cleaning and preprocessing; Divide the preprocessed data into several subsets; use part of the data subsets as training sets and the remaining data subsets as validation sets, and store the data subsets in a distributed manner on different computing nodes; Each computing node uses the training set to perform distributed training on the LLM, with the goal of minimizing the loss function of the LLM. When the maximum number of training rounds is reached, each computing node outputs a preliminarily trained LLM. Each round of distributed training includes the following process: (a) Each computing node reads at least one data subset in the training set; (b) Each computing node independently calculates the data gradient of the data subset; the data gradient is the partial derivative of the loss function of the LLM with respect to the parameters of the LLM; (c) Summarize data gradients; (d) Each computing node synchronously updates the parameters of the LLM based on the data gradient; Integrate the preliminarily trained LLMs in each computing node to obtain an integrated LLM; Use the validation set and evaluation indicators to evaluate the integrated LLM, adjust the training parameters of the LLM, and obtain the best LLM; Leverage the best of LLM for automated content creation.

[0027] The specific embodiments of the method are as follows: (1) Training data sharding Divide the training data into several subsets. The purpose of this step is to break down the large data set into smaller parts so that they can be processed in parallel on multiple computing nodes and reduce the burden on a single machine.

[0028] Dividing the training data into several subsets is a critical task. This process can not only improve data processing efficiency, but also optimize the utilization of computing resources, especially when processing large-scale data sets. The specific implementation process can be divided into the following steps: First, you need to obtain the original data set. The data set may come from multiple sources, such as databases, CSV files, or API interfaces. Data types may include text, images, audio, etc., depending on the application scenario. After obtaining the original data set, data cleaning is required. Data cleaning includes removing duplicate records, processing missing values ​​and outliers. Removing duplicate records helps reduce redundant data and ensure the uniqueness of data. There are many ways to handle missing values. For example, you can choose to delete records containing missing values, or use methods such as mean and interpolation to fill in missing values. The handling of outliers is equally important. Statistical methods or business rules can be used to identify and handle outliers to ensure the rationality and consistency of the data.

[0029] After data cleaning, the data needs to be standardized or normalized. Standardization refers to converting the data into a form with a mean of 0 and a variance of 1, which is particularly important for feature data with different dimensions. Normalization is scaling the data to a specific range (such as between 0 and 1), which has a significant effect on the training of models such as neural networks. Through standardization or normalization, the dimensional differences between features can be eliminated, making the model training process more stable and efficient.

[0030] After preprocessing, the dataset needs to be divided into features and labels. Features are the input data used by the model to learn, while labels are the target output of the model. For supervised learning tasks, the division of features and labels is a crucial step that directly affects the learning effect of the model.

[0031] The simplest data partitioning strategy is random partitioning, which randomly divides the data set into several subsets. This method is suitable for situations where the data is relatively uniform and has no obvious distribution characteristics, and the repeatability of the partition can be controlled by random seeds. For classification problems, stratified sampling is a common method. Stratified sampling can ensure that the proportion of each category in each subset is consistent with the original data set, avoiding the problem of data imbalance. For example, when dealing with classification problems containing multiple categories, stratified sampling can effectively maintain the representativeness of each category in each subset, thereby improving the generalization ability of the model.

[0032] In some cases, you may need to partition based on specific features. For example, for time series data, you can partition by time order; for geographic data, you can partition by geographic region. This approach ensures that each subset has a specific feature distribution, which is more in line with the needs of actual application scenarios. When designing a data partitioning strategy, it is usually necessary to divide the data into a training set and a validation set. The training set is used to train the model, while the validation set is used to evaluate the performance of the model. Reasonable division of the training set and validation set can prevent model overfitting and help select the best model parameters.

[0033] After determining the data partitioning strategy, the next step is to generate specific data subsets. You can use the cross-validation method to divide the data into several parts, each of which is a subset. Cross-validation can ensure that each subset can be used for training and verification, maximizing the utilization of data. After generating subsets, each subset needs to be saved separately for subsequent processing. When saving data subsets, you can use the directory structure in the file system or the partition table in the database to manage the subset data. Each subset needs to be assigned a unique identifier for subsequent access and processing.

[0034] In order to achieve parallel processing, the generated subsets need to be distributed and stored on different computing nodes. Distributed file systems (such as Hadoop HDFS) or cloud storage services (such as Amazon S3) can be used to manage these data subsets. Distributed file systems can provide storage solutions with high reliability, high availability, and high scalability. When storing data subsets, it is necessary to assign a unique identifier to each subset and upload it to the distributed storage system. This step ensures that in the subsequent distributed computing process, each node can independently access and process its own data subset. Scripts or automated tools can be used to batch upload and manage data subsets to ensure the consistency and reliability of data storage.

[0035] Common distributed computing frameworks include Apache Spark and Dask. These frameworks can process data in parallel on multiple nodes and support the processing and analysis of large-scale data sets. With the support of the distributed computing framework, each node reads one or more subsets and performs independent model training. Parallel training can make full use of the computing resources of multiple computers, greatly improving the speed and efficiency of training. Each node will train its own model separately and save the training results. During the training process, each node can independently load data, train models, and save results. Through the task scheduling and resource management functions of the distributed computing framework, the parallel computing tasks of multiple nodes can be effectively managed and coordinated to ensure the smooth progress of the training process.

[0036] After each node completes model training, all subset models need to be integrated. Common integration methods include model averaging and voting. These methods can combine the advantages of multiple models to improve the generalization ability and stability of the model. For example, the prediction results of multiple subset models can be averaged, or a voting mechanism can be used to select the final prediction result. After the model is integrated, it needs to be evaluated to ensure that its performance on the validation set meets expectations. Evaluation can use a variety of indicators, such as accuracy, precision, recall, etc., and select appropriate evaluation indicators according to specific tasks. Through model evaluation, the deficiencies of the model can be identified and corrected, and the model performance can be further optimized. According to the evaluation results, the data partitioning strategy, model training parameters and integration methods can be adjusted and optimized. In practical applications, each step can be continuously iterated and improved to achieve the best results.

[0037] The whole process starts from data preparation, goes through data preprocessing and subset generation, then to distributed storage and parallel training, and finally to model integration and evaluation, involving multiple steps and technologies. First, the quality of the data is ensured through data cleaning and preprocessing; then, a reasonable data partitioning strategy is designed to ensure the reasonable distribution of the data; next, data subsets are generated and distributed storage is performed to lay the foundation for parallel training; with the support of the distributed computing framework, each node trains the model in parallel, improving the training efficiency; finally, by integrating multiple subset models, the performance of the model is further improved.

[0038] In this process, the advantages of data partitioning and distributed computing are reasonably utilized, which not only ensures the training efficiency of the model, but also improves the performance of the model. Through parallel processing and model integration, the challenges of large-scale data sets can be effectively addressed, providing an efficient and reliable model training method. In practical applications, each step can be adjusted and optimized according to specific needs to achieve the best results.

[0039] (2) Parallel processing and distributed gradient descent Through the Distributed Gradient Descent algorithm, each node independently calculates the data gradient of its subset. Then, these gradients are aggregated among all nodes and the model parameters are updated synchronously. This method ensures that each node does not affect other nodes when processing its own data, thereby achieving efficient parallel computing.

[0040] Through the distributed gradient descent algorithm, each node independently calculates the data gradient of its subset, and then aggregates these gradients and synchronously updates the model parameters, which is an effective parallel computing method. This process ensures that each node does not affect other nodes when processing its own data, thereby achieving efficient parallel computing. The following is a detailed description of the process.

[0041] First, the raw data needs to be preprocessed and divided. The preprocessing steps include cleaning the data, handling missing values, and standardizing to ensure data quality. Next, the data set is divided into a training set and a validation set to evaluate the performance of the model. In order to further improve the efficiency of parallel computing, the training set will be divided into multiple subsets, each of which will be assigned to an independent computing node. These subsets can be generated by random partitioning or stratified sampling to ensure that each subset is representative.

[0042] Before training begins, model parameters need to be initialized. Model parameters are usually initialized with random values ​​or certain rules to ensure that all nodes have consistent model parameters in the initial state. This step is the basis for synchronous updates because all nodes need to start calculations from the same parameter starting point.

[0043] After receiving the assigned subset, each computing node starts to independently calculate the data gradient of the subset. The data gradient refers to the partial derivative of the model loss function with respect to the model parameters, which reflects how the model parameters need to be adjusted to minimize the loss. Each node independently processes the data of its subset and calculates the corresponding gradient. This process is parallel because each node only needs to focus on its own data subset and does not need to communicate with other nodes.

[0044] After each node has independently calculated the data gradient, all nodes need to aggregate the gradients they have calculated. Aggregation can be achieved through a central parameter server or a decentralized method. In the central parameter server method, each node sends the calculated gradient to the central server, which aggregates the gradients of all nodes and updates the model parameters. The decentralized method is that each node exchanges gradient information with other nodes, and finally reaches a consensus on the gradient aggregation result among all nodes.

[0045] After the gradients are aggregated, all nodes update the model parameters synchronously. This step ensures that each node uses the same model parameters at the beginning of the next iteration. The key to synchronous update is that all nodes need to update parameters at the same time to avoid training deviations caused by inconsistent parameters. Synchronous update is usually performed at the end of each iteration to ensure that each iteration performs gradient calculations based on the same parameters.

[0046] The above steps are repeated until the model converges or reaches the preset number of iterations. Each round of iteration includes independent calculation of data gradients, aggregation of gradients, and synchronous update of model parameters. Through multiple rounds of iterations, the model parameters are gradually optimized, and the goal of minimizing the loss function is finally achieved.

[0047] After the model training is completed, the model needs to be evaluated and verified. The data of the validation set is used to evaluate the performance of the model to ensure that the model has good generalization ability outside the training data. Evaluation indicators can include accuracy, precision, recall, etc., and appropriate evaluation indicators are selected according to specific tasks. Through evaluation, it can be determined whether the model has achieved the expected effect and whether further adjustment or optimization is needed.

[0048] Implementing a distributed gradient descent algorithm requires an efficient distributed computing system and architecture. Common distributed computing frameworks include Apache Spark, TensorFlow, PyTorch, etc. These frameworks provide powerful parallel computing capabilities and data processing tools to support the implementation of distributed gradient descent. The framework usually contains a parameter server or a decentralized communication mechanism to support gradient aggregation and parameter synchronization between nodes.

[0049] In the process of distributed gradient descent, communication and synchronization between nodes are key. The communication mechanism determines how gradient information is transmitted between nodes. Common communication mechanisms include message passing interface (MPI) and remote procedure call (RPC). The synchronization mechanism ensures that all nodes update parameters at the same time point to avoid training deviations caused by asynchrony. Frameworks usually provide built-in communication and synchronization mechanisms to simplify the implementation of distributed gradient descent.

[0050] In order to improve the efficiency of distributed gradient descent, performance optimization can be performed from multiple aspects. First, the data partitioning strategy can be optimized to ensure that the amount of data in each subset is balanced and the load difference between nodes is reduced. Second, the communication mechanism can be optimized to reduce the communication overhead by compressing gradient information and reducing the communication frequency. In addition, the hybrid parallel method can be used to combine the advantages of data parallelism and model parallelism to further improve the computational efficiency.

[0051] In terms of scalability, distributed gradient descent has good scalability and can handle larger data sets and more complex models by increasing the number of computing nodes. During the expansion process, attention should be paid to communication overhead and load balancing to ensure the overall performance of the system.

[0052] Through the distributed gradient descent algorithm, each node independently calculates the data gradient of its subset, then aggregates these gradients and synchronously updates the model parameters, achieving efficient parallel computing. This process includes data preparation and partitioning, initializing model parameters, independently calculating data gradients, aggregating gradients, synchronously updating model parameters, repeated iterative processes, model evaluation and verification, distributed systems and architectures, communication and synchronization mechanisms, and performance optimization and expansion. Through these steps, the distributed gradient descent algorithm can perform well in the training of large-scale data sets and complex models, providing an efficient and reliable parallel computing method.

[0053] (3) Mixed Precision Training Mixed Precision Training technology can speed up the training process and reduce memory usage by using 16-bit floating point numbers for calculations. This method can significantly improve computing efficiency without significantly affecting model accuracy.

[0054] Mixed precision training is a technique that speeds up the training process and reduces memory usage by using 16-bit floating point numbers for calculations. This method significantly improves computational efficiency without significantly affecting model accuracy. The following is the specific implementation process.

[0055] The core of mixed precision training is to convert some calculations from 32-bit floating point numbers to 16-bit floating point numbers. 32-bit floating point numbers (FP32) are widely used in deep learning because they have enough precision to handle most calculations. However, 32-bit floating point numbers require larger storage space and computing resources. 16-bit floating point numbers (FP16) use less storage space and computing resources, but have lower precision. Mixed precision training combines the advantages of both: using FP16 for most calculations without losing too much precision, while keeping FP32 in key steps to ensure computational stability and accuracy.

[0056] Mixed precision training requires hardware support, and modern graphics processing units (GPUs), such as NVIDIA's TensorCores, are specifically designed to perform FP16 calculations efficiently. Before enabling mixed precision training, it is critical to confirm that the hardware you are using supports this feature.

[0057] Deep learning frameworks such as TensorFlow and PyTorch provide support for mixed precision training. These frameworks simplify the implementation of mixed precision training, allowing developers to easily enable and manage FP16 calculations in their code.

[0058] First, the raw data is preprocessed, including cleaning, standardization, and data set partitioning. This step ensures data quality and lays the foundation for subsequent training. The steps of data preparation are the same as traditional full-precision training, with the focus on ensuring that the data is clean and standardized before entering the training process.

[0059] In mixed precision training, the model is initialized in the same way as full precision training. The model parameters need to be initialized and the model is set to a mode suitable for mixed precision calculations. In frameworks such as TensorFlow and PyTorch, mixed precision training can be enabled by setting the precision mode of the model and optimizer.

[0060] When enabling mixed precision training, you need to set the precision mode of the model and optimizer. In TensorFlow, you can use the mixed precision API to set the model's calculation mode to mixed precision. In PyTorch, you can use the automatic mixed precision (AMP) package to manage the switch between FP16 and FP32 calculations. These APIs and tools automatically handle precision conversion, eliminating the need for developers to manually adjust the calculation precision of each step.

[0061] During model training, forward propagation and back propagation calculations will be partially performed using FP16. Forward propagation refers to the process of input data passing through the model to calculate the output, and back propagation is the process of adjusting model parameters based on the error between the output and the target value. In mixed precision training, most of the forward propagation calculations use FP16, while gradient calculations and key accumulation operations use FP32 to ensure accuracy and stability.

[0062] Due to the small representation range of FP16, numerical underflow may occur during training. To prevent this, dynamic loss scaling is used. Dynamic loss scaling scales the loss value to make it within the representation range of FP16, and then scales it back before updating the model parameters. This step ensures numerical stability during training by automatically adjusting the loss scaling factor.

[0063] At the end of each training step, the calculated gradients are accumulated into FP32 variables. This avoids gradient update errors due to precision issues. After the accumulation is complete, the model parameters are updated based on the accumulated gradients. The optimizer manages this process to ensure accuracy and stability when updating parameters.

[0064] During the training process, the model is evaluated and validated regularly to ensure that the performance of the model on the validation set meets expectations. The evaluation steps are the same as traditional training, including calculating the loss of the validation set and various performance indicators such as accuracy, precision, and recall. Through evaluation, the impact of mixed precision training on model accuracy can be monitored to ensure that it does not significantly lose model performance while improving computational efficiency.

[0065] The mixed precision training process requires constant adjustment and optimization. According to the performance and accuracy changes observed during training, hyperparameters such as loss scaling factor and learning rate are adjusted. Parameters such as batch size can also be adjusted according to hardware performance and data set size to achieve the best training effect.

[0066] Mixed precision training effectively speeds up the training process and reduces memory usage by using 16-bit floating point numbers for calculations. In the implementation process, you first need to ensure that the hardware and software framework support mixed precision calculations, and then set the calculation mode of the model and optimizer to mixed precision by enabling the mixed precision API. During the training process, dynamic loss scaling and gradient accumulation techniques are used to ensure numerical stability and calculation accuracy. By regularly evaluating and adjusting the training process, the model can maintain good performance while being efficiently trained.

[0067] (4) GPU cluster virtualization and device grid management Use the Ray framework and Alpa architecture to virtualize GPU clusters and manage device meshes. Ray provides a task and actor model that can execute tasks asynchronously on different machines in the cluster. Alpa implements complex parallel computing and resource allocation through automatic partitioning and device mesh management.

[0068] Using the Ray framework and Alpa architecture for GPU cluster virtualization and device grid management is an efficient distributed computing method. Ray provides a task and actor model that can execute tasks asynchronously on different machines in the cluster. Alpa implements complex parallel computing and resource allocation through automatic partitioning and device grid (DeviceMesh) management. The following is the specific implementation process.

[0069] In modern deep learning applications, training large-scale models requires a lot of computing resources, especially GPU clusters. However, managing and allocating these resources for efficient parallel computing is a complex task. The Ray framework and Alpa architecture provide a set of solutions to achieve efficient resource utilization and task scheduling through GPU cluster virtualization and device grid management.

[0070] Ray is a distributed computing framework designed to simplify the development of distributed applications. It provides a task and actor model that allows developers to easily execute tasks asynchronously on different machines in a cluster. Ray's core features include efficient task scheduling, automatic fault tolerance, and support for large-scale clusters. With Ray, developers can distribute computing tasks to multiple nodes and manage the execution status of these tasks.

[0071] Alpa is a distributed computing system for deep learning that implements complex parallel computing and resource allocation through automatic partitioning and device mesh management. Alpa's device mesh management mechanism allows developers to define the distribution of computing tasks on different devices, thereby optimizing resource utilization and computing efficiency. Alpa also supports automatic partitioning to break down large-scale computing tasks into smaller subtasks for parallel execution on different devices.

[0072] In practical applications, GPU cluster virtualization and device grid management using Ray and Alpa can be achieved by following the steps below: First, configure and initialize the GPU cluster. Make sure that the necessary software is installed on each node, including Ray and Alpa. When configuring a Ray cluster, you need to specify the IP address and resource information (such as the number of GPUs and memory size) of each node in the cluster. Through Ray's cluster startup script, you can start the entire cluster and ensure that each node can communicate with each other.

[0073] In the Ray framework, tasks and actor models are defined to distribute computing tasks in a cluster. Tasks are the most basic computing unit in Ray and can be executed asynchronously on different nodes. Actors are computing units with states that can be shared between multiple tasks. In the implementation process, the training steps of a deep learning model can be defined as Ray tasks, and the parameters and states of the model can be managed through actors.

[0074] Using Alpa's automatic partitioning function, large-scale computing tasks can be broken down into smaller subtasks. Alpa can automatically decide how to divide tasks to different devices for execution based on the characteristics of the computing tasks and the resources of the cluster. Device Mesh is Alpa's core mechanism for managing computing resources. By defining a device mesh, you can specify the distribution of computing tasks on different devices. The configuration of the device mesh can be adjusted according to specific computing needs to ensure optimal resource utilization.

[0075] Through Ray's task scheduling function, computing tasks are distributed to different nodes for execution. Ray's efficient scheduling algorithm ensures that tasks can be assigned to appropriate computing resources in the shortest possible time. The asynchronous execution feature allows computing tasks to be performed in parallel, thereby improving overall computing efficiency. During task execution, Ray will automatically handle node failures and recovery to ensure that tasks can be completed smoothly.

[0076] During task execution, the execution status and resource utilization of tasks are monitored in real time. Ray provides a variety of monitoring tools to help developers track the progress of tasks and the resource usage of the cluster. Based on the monitoring results, the task allocation strategy and device grid configuration are adjusted to further optimize computing performance and resource utilization.

[0077] After completing resource configuration and task scheduling, the actual model training begins. Using Ray's task model and Alpa's device grid management, the deep learning model training process can be efficiently executed. During the training process, Ray's monitoring tools are used to observe the progress and performance of the training in real time to ensure that the training status of the model is consistent on each node. After the training is completed, the model is verified, its performance is evaluated, and necessary adjustments are made.

[0078] Using the Ray framework and Alpa architecture for GPU cluster virtualization and device grid management is an efficient distributed computing method. Through Ray's task and actor model, tasks can be executed asynchronously on different machines in the cluster to ensure efficient task scheduling and resource utilization. Alpa achieves complex parallel computing and resource allocation through automatic partitioning and device grid management, further improving computing efficiency.

[0079] In the specific implementation process, first configure and initialize the GPU cluster to ensure that each node can communicate with each other. Then, define the computing tasks and Actor models and distribute them to different nodes for execution through Ray. Using Alpa's automatic partitioning and device grid management functions, large-scale computing tasks are broken down into smaller subtasks and assigned to different devices for parallel computing. Through real-time monitoring and optimization, efficient execution of tasks and optimal utilization of resources are ensured. In the actual model training process, the combination of Ray and Alpa can significantly improve training efficiency and model performance.

[0080] (5) Optimizing collective communication operations In distributed training, collective communication operations (such as AllGather) may become a bottleneck. By using AWS's SMDDP library instead of the traditional NCCL library, these operations can be optimized and the overall training efficiency can be improved. SMDDP reduces communication overhead and improves computing efficiency through more efficient network communication modes and GPU resource management.

[0081] In distributed training, collective communication operations such as AllGather may become a bottleneck. These operations require transferring a large amount of data between multiple GPUs, which can seriously affect the overall training efficiency if not handled properly. To optimize these operations, AWS's SMDDP library can be used instead of the traditional NCCL library. SMDDP reduces communication overhead and improves computing efficiency through more efficient network communication modes and GPU resource management. The following is the specific implementation process.

[0082] In distributed deep learning training, collective communication operations are essential, especially in multi-GPU or multi-node environments. Common collective communication operations include AllReduce, AllGather, Broadcast, etc. These operations require data to be transferred between different computing devices to ensure the consistency of model parameters. However, the traditional NCCL (NVIDIA Collective Communications Library) library may encounter communication bottlenecks when dealing with large-scale clusters and high-concurrency tasks, affecting the overall training efficiency.

[0083] AWS's SMDDP (Synchronous Multi-device Data Parallelism) library is an optimized distributed training library specifically designed to improve the efficiency of collective communication operations. SMDDP reduces communication overhead through more efficient network communication modes and GPU resource management. It leverages AWS's high-performance network infrastructure to optimize data transmission paths and communication protocols, thereby improving training efficiency.

[0084] First, you need to install the SMDDP library on each node participating in the training. Make sure that the version installed on all nodes is consistent and configure the corresponding dependency environment. You can get detailed installation steps and configuration guides from the AWS official documentation. After the installation is complete, you need to configure the cluster network to ensure that high-speed communication connections between nodes have been established and that the configuration files are correct.

[0085] Before starting training, initialize the distributed training environment. You need to set basic information about the cluster, including the number of nodes, the number of GPUs on each node, and the network topology. When using the SMDDP library, the initialization steps are similar to those of traditional NCCL, but additional SMDDP-related configuration parameters need to be specified. After initialization, ensure that all nodes can correctly identify and communicate.

[0086] In distributed training, data parallelism is usually used, that is, each node or GPU processes a different subset of data while updating the same model parameters. When configuring model and data parallelism, it is necessary to ensure the consistency of the model on each node and divide the data into multiple subsets so that they can be processed in parallel on different nodes. The SMDDP library provides some tools to help developers configure model and data parallelism more conveniently.

[0087] In the actual training code, replace the traditional NCCL collective communication operations with the corresponding operations provided by the SMDDP library. Replace the AllReduce operation of NCCL with the AllReduce operation of SMDDP. The API design of the SMDDP library is as compatible as possible with NCCL, making the replacement process relatively simple. During the replacement process, ensure that all collective communication operations use the SMDDP library correctly to give full play to its optimization effect.

[0088] SMDDP improves the efficiency of collective communication operations by optimizing network communication patterns. When configuring and using the SMDDP library, you can further optimize network communication. Adjust the batch size of data transmission, optimize the data transmission path, and reduce the latency of data transmission. AWS provides some tools and configuration options to help developers optimize network communication settings to achieve the best communication efficiency.

[0089] In addition to optimizing communication operations, SMDDP also improves GPU resource management and scheduling. Through smarter resource allocation strategies, SMDDP can more effectively utilize GPU computing resources and reduce resource waste. When configuring and using SMDDP, you can use these optimization strategies to ensure that the computing resources of each GPU are fully utilized and improve overall training efficiency.

[0090] During the training process, the progress and performance indicators of the training are monitored in real time to ensure the efficiency of collective communication operations and the effectiveness of model training. The SMDDP library provides some monitoring tools that can help developers track and analyze the performance of communication operations and identify possible bottlenecks and problems. According to the monitoring results, the training configuration and communication parameters can be adjusted in time to further optimize the training process.

[0091] After training is complete, the model is evaluated and validated. The validation set is used to evaluate the performance of the model to ensure that it achieves the expected results. During the evaluation process, special attention is paid to the efficiency of collective communication operations and their impact on the overall training time. By comparing the training results and efficiency before and after using the SMDDP library, the optimization effect of SMDDP can be evaluated.

[0092] Using AWS's SMDDP library to replace the traditional NCCL library can significantly optimize the collective communication operations in distributed training and improve the overall training efficiency. In the specific implementation process, first install and configure the SMDDP library to initialize the cluster and distributed training environment. Then, replace the collective communication operations in the training code with the corresponding operations provided by the SMDDP library, and optimize network communication and resource management. During the training process, monitor the training progress and performance indicators in real time, and adjust the configuration in time to achieve the best results. Finally, through model evaluation and verification, ensure that the model maintains good performance while training efficiently.

[0093] Embodiment 2

[0094] The present invention proposes a distributed training system of LLM corresponding to the method of embodiment 1, comprising: Data acquisition module, used to collect data and perform data cleaning and preprocessing; The data partitioning and storage module is used to divide the preprocessed data into several subsets; a part of the data subsets are used as training sets, and the remaining data subsets are used as validation sets, and the data subsets are distributedly stored on different computing nodes; GUP cluster, each computing node in the cluster is a GPU, the network topology of the GUP cluster is as follows Figure 2 As shown in the figure, each computing node uses the training set to perform distributed training on the LLM, with the goal of minimizing the loss function of the LLM. When the maximum number of training rounds is reached, each computing node outputs a preliminarily trained LLM; each round of distributed training includes the following process: (a) Each computing node reads at least one data subset in the training set; (b) Each computing node independently calculates the data gradient of the data subset; the data gradient is the partial derivative of the loss function of the LLM with respect to the parameters of the LLM; (c) Summarize data gradients; (d) Each computing node synchronously updates the parameters of the LLM based on the data gradient; The model integration module is used to integrate the preliminarily trained LLMs in each computing node to obtain an integrated LLM; The model adjustment module is used to evaluate the integrated LLM using the validation set and the evaluation index, adjust the training parameters of the LLM, and obtain the best LLM; the best LLM is used for automatic content creation.

[0095] The implementation methods of each module and module function in the system are completely consistent with the steps of the method in Example 1, so they will not be repeated here.

[0096] Embodiment 3

[0097] The present invention proposes an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the distributed training method of the LLM as described in the first embodiment is implemented.

[0098] Embodiment 4

[0099] The present invention provides a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the distributed training method of LLM as described in the first embodiment.

[0100] The distributed training method, system, device and storage medium of LLM of the present invention can be applied to various scenarios that require processing and generating natural language, including automated content creation, intelligent customer service, speech recognition and medical text analysis. These applications will significantly benefit from the model's efficient and accurate processing capabilities on different data subsets.

[0101] The present invention can not only improve model performance and computing efficiency and reduce training costs, but can also be applied to a wide range of business scenarios, such as intelligent customer service, automated content creation, and targeted advertising, thereby improving the service quality and market competitiveness of enterprises.

[0102] In the embodiments disclosed in the present application, the computer storage medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or apparatus. The computer storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. More specific examples of computer storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0103] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0104] The above are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.

Claims

1. A distributed training method for LLM, characterized in that: The following steps are involved: Collect data and perform data cleaning and preprocessing; Divide the preprocessed data into several subsets; use part of the data subsets as training sets and the remaining data subsets as validation sets, and store the data subsets in a distributed manner on different computing nodes; Each computing node uses the training set to perform distributed training on the LLM, with the goal of minimizing the loss function of the LLM. When the maximum number of training rounds is reached, each computing node outputs a preliminarily trained LLM. Each round of distributed training includes the following process: (a) Each computing node reads at least one data subset in the training set; (b) Each computing node independently calculates the data gradient of the data subset; the data gradient is the partial derivative of the loss function of the LLM with respect to the parameters of the LLM; (c) Summarize data gradients; (d) Each computing node synchronously updates the parameters of the LLM based on the data gradient; Integrate the preliminarily trained LLMs in each computing node to obtain an integrated LLM; Use the validation set and evaluation indicators to evaluate the integrated LLM, adjust the training parameters of the LLM, and obtain the best LLM; Leverage the best of LLM for automated content creation.

2. The distributed training method of LLM according to claim 1, characterized in that: The method of dividing the pre-processed data into several subsets is a cross-validation method; The distributed storage of data subsets on different computing nodes specifically includes: using a distributed file system or a cloud storage service to perform distributed storage of data subsets, when storing data subsets, assigning a unique identifier to each data subset, and uploading it to the distributed storage system; Use scripts or automation tools to bulk upload and manage subsets of data.

3. The distributed training method of LLM according to claim 1, characterized in that: The method of aggregating data gradients is implemented through a central parameter server or a decentralized method; In the central parameter server approach, each compute node sends the calculated gradients to the central server, which aggregates the gradients of all compute nodes and updates the LLM parameters; The decentralized approach is that each node exchanges gradient information with other nodes. The communication mechanism between nodes includes message passing interface and remote procedure call, and finally a consistent gradient summary result is reached among all nodes.

4. The distributed training method of LLM according to claim 1, characterized in that: During distributed training, the deep learning framework used is TensorFlow or PyTorch, and mixed precision training is enabled by setting the precision mode of the model and optimizer; If you use TensorFlow, use the mixed precision API to set the model's computation mode to mixed precision. If you use PyTorch, use the automatic mixed precision package to manage the switch between 16-bit floating point and 32-bit floating point computations. In the process of distributed training of LMM, the forward propagation and back propagation calculations will be partially performed using 16-bit floating point numbers; forward propagation refers to the process of input data being calculated through LLM output, and back propagation is the process of adjusting LMM parameters based on the error between the output and the target value; in mixed precision training, most of the forward propagation calculations use 16-bit floating point numbers, while gradient calculations and key accumulation operations use 32-bit floating point numbers; Use dynamic loss scaling to scale the loss function value to within the 16-bit floating point representation range, and then scale it back before updating the LMM parameters. At the end of each training step, the calculated gradients are accumulated into 32-bit floating point variables, and the accumulated gradients are used to update the parameters of the LMM.

5. The distributed training method of LLM according to claim 1, characterized in that: The computing node is a GPU cluster. Before the distributed training of LLM, the process of GPU cluster virtualization and device grid management using the Ray framework and Alpa architecture is also included. The specific implementation steps are: First, configure and initialize the GPU cluster and ensure that Ray and Alpa are installed on each computing node. When configuring a GPU cluster with Ray installed, specify the IP address and resource information of each node in the cluster. Start the entire cluster using the GPU cluster startup script with Ray installed and ensure that each node can communicate with each other. Then, define computing tasks and Actor models in the Ray framework. Tasks are the most basic computing units in Ray and can be executed asynchronously on different nodes. Actor is a computing unit with state that can share state between multiple tasks. Computing tasks are distributed to different nodes for execution through Ray. Using Alpa's automatic partitioning and device grid management capabilities, large-scale computing tasks can be broken down into smaller subtasks and assigned to different nodes in the GPU cluster for parallel computing.

6. The distributed training method of LLM as claimed in claim 5, characterized in that: Each node in the GPU cluster is also installed with the SMDDP library to improve the efficiency of cluster communication during distributed training; In the distributed training process based on the SMDDP library, the tools provided by the SMDDP library are used to enable the nodes of the GPU cluster to process different data subsets in parallel, update the parameters of the LLM synchronously, and adjust the batch size and data transmission path of the data transmission; Use the monitoring tools provided by the SMDDP library to monitor the progress and performance indicators of training in real time, and adjust the training configuration and communication parameters in a timely manner.

7. The distributed training method of LLM as claimed in claim 5, characterized in that: The device grid is the core mechanism of the Alpa framework for managing computing resources. By defining the device grid, the distribution of computing tasks on different devices is specified. The configuration of the device grid is adjusted according to specific computing needs. Specifically, the progress of tasks and the resource usage of the cluster are tracked through the monitoring tool of the Ray framework. According to the monitoring results, the task allocation strategy and the configuration of the device grid are adjusted.

8. A distributed training system for LLM, characterized in that: include: Data acquisition module, used to collect data and perform data cleaning and preprocessing; The data partitioning and storage module is used to divide the preprocessed data into several subsets; a part of the data subsets are used as training sets, and the remaining data subsets are used as validation sets, and the data subsets are distributedly stored on different computing nodes; GUP cluster, each computing node in the cluster is a GPU. Each computing node uses the training set to perform distributed training on LLM, with the goal of minimizing the loss function of LLM. When the maximum number of training rounds is reached, each computing node outputs a preliminarily trained LLM; each round of distributed training includes the following process: (a) Each computing node reads at least one data subset in the training set; (b) Each computing node independently calculates the data gradient of the data subset; the data gradient is the partial derivative of the loss function of the LLM with respect to the parameters of the LLM; (c) Summarize data gradients; (d) Each computing node synchronously updates the parameters of the LLM based on the data gradient; The model integration module is used to integrate the preliminarily trained LLMs in each computing node to obtain an integrated LLM; The model adjustment module is used to evaluate the integrated LLM using the validation set and the evaluation index, adjust the training parameters of the LLM, and obtain the best LLM; the best LLM is used for automatic content creation.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the distributed training method for LLM as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: The computer program enables a computer to execute the distributed training method for LLM according to any one of claims 1 to 7.

Images