Model training method of algorithm based on specified GPU thread

By clearly specifying the combination of GPU threads, memory management, data transmission optimization and parallelized training algorithms, challenges such as GPU parallelism in deep learning model training are solved, which significantly improves training efficiency and performance, and is suitable for various deep learning tasks.

CN119150942APending Publication Date: 2024-12-17UNICLOUD TECH CO LTD
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411158898.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-22
Publication Date
2024-12-17

AI Technical Summary

Technical Problem

Deep learning model training has challenges such as parallel utilization, memory requirements, data transmission overhead and training algorithm parallelism on GPUs, resulting in resource waste and inefficient performance, especially on large models and large-scale data sets, which are difficult to effectively accelerate convergence.

Method used

By clearly specifying the working methods of GPU threads, GPU thread management strategies, memory optimization and data transmission optimization technologies are used to realize parallel training algorithms and perform automatic performance adjustments to ensure the best training results under different hardware configurations and task requirements.

Benefits of technology

It significantly improves the efficiency and performance of deep learning model training, reduces resource waste, solves problems such as GPU parallelism, memory management, data transmission and training algorithm parallelism, and is suitable for various deep learning tasks and data sets of different scales.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119150942A_ABST
    Figure CN119150942A_ABST
Patent Text Reader

Abstract

The invention provides a model training method of an algorithm based on a specified GPU thread. The method comprises the following steps: selecting GPU hardware and a deep learning framework; designing and defining the structure and parameters of a deep learning model to prepare for subsequent training; preparing training data, loading the training data into a GPU memory, and providing required data input for model training; a GPU thread management strategy is adopted, and it is guaranteed that a user accurately controls the number, distribution, scheduling and priority of GPU threads in the training process so as to optimize parallelism; through the memory management and data transmission optimization technology, the data transmission between the GPU and the CPU is reduced, and the memory occupation during the training period is reduced. The method has the beneficial effects that a series of challenges related to GPU parallel computing are solved through innovations in the aspects of clearly specifying GPU threads, memory management, data transmission optimization, a parallel training algorithm, automatic performance adjustment and the like, and the efficiency and the performance of deep learning model training are expected to be remarkably improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of models, and in particular relates to a model training method based on an algorithm of a specified GPU thread. Background Art

[0002] Deep learning models, AIGC training models have achieved great success in computer vision, natural language processing, speech recognition and other fields, but their training requires a lot of computing resources and time. GPUs (graphics processing units) have become a key tool for deep learning training because they have powerful parallel computing capabilities and can significantly accelerate the model training process. However, performing deep learning model training on GPUs is still challenging, especially for large models and large-scale data sets: 1. GPU parallelism utilization: Modern GPUs have a large number of processing units, but to fully utilize their parallelism, the allocation of threads and tasks needs to be carefully managed. Traditional training methods may not be able to effectively utilize the full performance of the GPU, resulting in resource waste; 2. Memory requirements for large models: Large deep learning models require a lot of memory to store model parameters and intermediate calculation results; 3. Data transfer overhead: The data transfer overhead between transferring data from host memory to GPU memory and vice versa may become a bottleneck; 4. Parallelism of training algorithms: Deep learning training algorithms usually require a large number of iterations and calculations. How to effectively parallelize these computing tasks to accelerate the convergence rate is an important technical challenge; 5. Performance optimization: In practical applications, the training of deep learning models usually needs to be completed in a limited time; Summary of the invention

[0003] In view of this, the present invention aims to propose a model training method based on an algorithm for specifying GPU threads, aiming to solve problems in GPU parallelism, memory management, data transmission, training algorithm parallelism, and performance optimization. By explicitly specifying the working mode of GPU threads, this method is expected to improve the efficiency of deep learning model training, accelerate the training process, and reduce resource waste, thus having broad application prospects in the field of deep learning.

[0004] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0005] In a first aspect, the present solution discloses a model training method based on an algorithm of a specified GPU thread, comprising:

[0006] Choose GPU hardware and framework: Choose GPU hardware and deep learning framework;

[0007] Model design and definition: Design and define the structure and parameters of the deep learning model to prepare for subsequent training;

[0008] Data preparation and loading: Prepare training data and load it into GPU memory to provide the required data input for model training;

[0009] GPU thread management: GPU thread management strategy is adopted to ensure that users can accurately control the number, distribution, scheduling and priority of GPU threads during training to optimize parallelism;

[0010] Memory optimization and data transfer: Through memory management and data transfer optimization technology, the data transfer between GPU and CPU is reduced, and the memory usage during training is reduced;

[0011] Parallelized training algorithm: Implement parallelized training algorithm so that computing tasks can be processed simultaneously on multiple GPU threads, including parallel update of model parameters and distributed processing of gradient calculation, to accelerate the training process;

[0012] Automatic performance adjustment: Dynamically optimizes training performance based on current GPU hardware and workload, ensuring the best training results under different hardware configurations and task requirements.

[0013] Furthermore, the data preparation and loading includes:

[0014] Data preprocessing: including data cleaning, normalization, and enhancement to improve the generalization ability of the model;

[0015] Data batching: Divide the data into multiple small batches, and input the data of each batch into the model for training at the same time;

[0016] Data loading: Use multi-threaded or multi-process data loading mechanisms to speed up data reading and load data into GPU memory;

[0017] In the data batching, the batch size is B, the total data volume is N, and the data volume of each batch is:

[0018]

[0019] Furthermore, the GPU thread management includes:

[0020] Thread quantity control: Dynamically adjust the number of concurrent threads to ensure maximum computing efficiency while avoiding thread competition and resource waste;

[0021] Thread scheduling strategy: By setting thread priorities, key computing tasks are prioritized and the resource usage of low-priority tasks is reduced;

[0022] Thread binding and core affinity: Bind threads to specific GPU cores to reduce thread switching and cross-core communication overhead;

[0023] in,

[0024] The number of threads is Nt, and the priority weight of the task is Wt. The resource ratio Rt obtained by each thread is expressed as:

[0025]

[0026] Furthermore, the memory optimization and data transmission include:

[0027] Memory reuse: During training, the buffers in GPU memory are reused to reduce dynamic memory allocation and release operations and avoid fragmentation.

[0028] Data pipeline and cache: Pre-fetch the next batch of data while the GPU is performing computing tasks and store it in the cache to achieve parallelization of computing and data loading;

[0029] Compression and decompression: compress data before transmission to reduce transmission time, and decompress quickly on the GPU side;

[0030] in,

[0031] The data transmission time before compression is Td, and the compression ratio is C. Then the transmission time after optimization is ToptT:

[0032]

[0033] Furthermore, the parallel training algorithm includes:

[0034] Data parallelism: The training data set is divided into multiple subsets, which are trained independently on different GPUs. After each GPU calculates the gradient, it is aggregated and the model parameters are updated.

[0035] Model parallelism: Divide different parts of the model into different GPUs for computing, which is suitable for training large models;

[0036] Distributed training: Model training is performed on multiple machines, with multiple GPUs running on each machine, and gradient synchronization and parameter updates are performed through efficient network communication protocols.

[0037] Furthermore, the model parallelism includes:

[0038] Data parallelism: Let the training data set be D, containing N samples. Use k GPUs for parallel training and divide the data set into k subsets. The amount of data in each subset is:

[0039]

[0040] For each subset Di, the loss function of the jth iteration is:

[0041]

[0042] Where θ(j) is the model parameter of the jth iteration, L is the loss function, and f(x;θ) is the output of the model;

[0043] Each GPU calculates the gradient of the corresponding subset:

[0044]

[0045] After all GPUs have calculated the gradients, they use the gradient synchronization mechanism to aggregate all gradients and update the model parameters:

[0046]

[0047] Where η is the learning rate;

[0048] Model Parallelism:

[0049] Assume that the model is divided into k parts and calculated on k GPUs respectively. For the i-th GPU, it is responsible for calculating the output of the i-th part of the model:

[0050] H i =f i (H i-1 θ i );

[0051] Among them, Hi is the output of the i-th layer, Hi-1 is the output of the previous layer, and θi is the parameter of the i-th layer; during the back propagation process, the calculation of the gradient of each part is as follows:

[0052]

[0053] at the same time, Passed to the previous layer via the chain rule:

[0054]

[0055] When updating parameters, after gradient aggregation, use synchronous update or asynchronous update:

[0056]

[0057] Distributed training:

[0058] Suppose there are M machines, each with km GPUs, then the total number of GPUs is

[0059] The size of the data subset on each machine is:

[0060]

[0061] Each subset is further divided into corresponding GPUs. Dm,i is the data subset processed by the i-th GPU on the m-th machine:

[0062]

[0063] The global parameter update formula is:

[0064]

[0065] Furthermore, the automatic performance adjustment includes:

[0066] Adaptive learning rate adjustment: Dynamically adjust the learning rate based on the training error or loss function value to avoid overfitting or non-convergence of training;

[0067] Load balancing and resource allocation: monitor the utilization of each GPU in real time and dynamically adjust the allocation of computing tasks to balance the load of all GPUs, thereby improving overall computing efficiency;

[0068] Performance monitoring and feedback tuning: By real-time monitoring of key indicators such as CPU, GPU utilization, memory usage, and network bandwidth during training, training parameters and scheduling strategies are automatically adjusted based on feedback.

[0069] In a second aspect, the present solution discloses an electronic device, comprising a processor and a memory that is communicatively connected to the processor and is used to store executable instructions of the processor, wherein the processor is used to execute a model training method based on an algorithm of a specified GPU thread as described in the first aspect.

[0070] In the third aspect, the present solution discloses a server, comprising at least one processor and a memory communicatively connected to the processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the processor so that the at least one processor executes the model training method based on an algorithm of a specified GPU thread as described in the first aspect.

[0071] In a fourth aspect, the present solution discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the model training method based on an algorithm of a specified GPU thread as described in the first aspect.

[0072] Compared with the prior art, the model training method based on the algorithm of specifying GPU thread described in the present invention has the following beneficial effects:

[0073] The model training method based on an algorithm for specifying GPU threads described in the present invention solves a series of challenges related to GPU parallel computing by clearly specifying GPU threads, memory management, data transmission optimization, parallel training algorithms, and automatic performance adjustment, and is expected to significantly improve the efficiency and performance of deep learning model training. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings:

[0075] Figure 1 A schematic diagram of a model training method based on an algorithm for specifying GPU threads according to an embodiment of the present invention. DETAILED DESCRIPTION

[0076] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0077] The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0078] The present invention provides a model training method based on a specified GPU thread, aiming to solve problems in GPU parallelism, memory management, data transmission, training algorithm parallelism, and performance optimization. The method comprises the following steps:

[0079] 1. Select GPU hardware and framework: Select appropriate GPU hardware and deep learning framework, ensuring support for clear definition and management of GPU threads.

[0080] 2. Model design and definition: Define the structure and parameters of the deep learning model.

[0081] 3. Data preparation: Prepare training data and load it into GPU memory.

[0082] 4. Explicit GPU thread management: This method provides a way to explicitly specify GPU threads, enabling users to precisely control the parallelism during model training. This includes specifying the number, distribution, scheduling, and priority of threads.

[0083] 5. Memory Optimization and Data Transfer: The invention may include a set of memory management and data transfer optimization techniques. These techniques are designed to minimize the transfer of data between the GPU and the CPU, as well as the memory usage during training. This helps improve performance and reduce resource consumption.

[0084] 6. Parallelized training algorithm: The method may include a parallelized training algorithm that can process computing tasks on multiple GPU threads simultaneously. This includes parallel updating of model parameters, distributed processing of gradient calculations, and other techniques to accelerate the training process.

[0085] 7. Automatic performance adjustment: The invention may have an automatic performance adjustment function that can dynamically optimize the performance of model training based on the GPU hardware and workload conditions. This helps to adapt to different hardware configurations and task requirements.

[0086] 8. Scalability and applicability: This invention may be designed to be scalable and applicable to various deep learning tasks and datasets of different sizes. It can run on a variety of GPU hardware and provide efficient model training solutions for different application scenarios.

[0087] The innovations include the following:

[0088] Method for explicitly specifying GPU threads: This invention introduces a method for explicitly specifying GPU threads, allowing users to precisely control the number of threads on the GPU and how they work. This precision can help address challenges in GPU parallelism, thereby improving the efficiency of deep learning model training.

[0089] Memory management and data transfer optimization: This invention may include a set of memory management and data transfer optimization techniques to reduce the frequent transfer of data between the GPU and the CPU, thereby reducing memory usage and improving performance. This is a key innovation in deep learning because data transfer and memory management are often performance bottlenecks.

[0090] Parallelized training algorithm: The invention may include a parallelized training algorithm that can fully utilize the multi-core computing power of the GPU. By distributing the training tasks to multiple GPU threads, more efficient model training can be achieved.

[0091] Automatic performance tuning: The invention may have an automatic performance tuning function that can automatically optimize performance based on the GPU hardware configuration and workload. This automatic tuning can make the method applicable to various hardware and task requirements.

[0092] Scalability and Applicability: This invention is designed to be scalable and adaptable to a variety of deep learning tasks and datasets of different sizes. This increases its applicability and gives it a wide range of potential uses in various deep learning application scenarios.

[0093] In general, the innovation of this invention is that it provides a comprehensive solution, which solves a series of challenges related to GPU parallel computing by clearly specifying GPU threads, memory management, data transfer optimization, parallel training algorithms, and automatic performance adjustment, and is expected to significantly improve the efficiency and performance of deep learning model training. These innovations make this method have important technical value and application prospects in the field of deep learning.

[0094] Those of ordinary skill in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0095] In the several embodiments provided in the present application, it should be understood that the disclosed methods and systems can be implemented in other ways. For example, the division of the units described above is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. The above-mentioned units may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present invention.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention.

[0097] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A model training method based on an algorithm for specifying GPU threads, characterized in that: include: Choose GPU hardware and framework: Choose GPU hardware and deep learning framework; Model design and definition: Design and define the structure and parameters of the deep learning model to prepare for subsequent training; Data preparation and loading: Prepare training data and load it into GPU memory to provide the required data input for model training; GPU thread management: GPU thread management strategy is adopted to ensure that users can accurately control the number, distribution, scheduling and priority of GPU threads during training to optimize parallelism; Memory optimization and data transfer: Through memory management and data transfer optimization technology, the data transfer between GPU and CPU is reduced, and the memory usage during training is reduced; Parallelized training algorithm: Implement parallelized training algorithm so that computing tasks can be processed simultaneously on multiple GPU threads, including parallel update of model parameters and distributed processing of gradient calculation, to accelerate the training process; Automatic performance adjustment: Dynamically optimizes training performance based on current GPU hardware and workload, ensuring the best training results under different hardware configurations and task requirements.

2. The model training method based on an algorithm of a specified GPU thread according to claim 1, characterized in that: The data preparation and loading include: Data preprocessing: including data cleaning, normalization, and enhancement to improve the generalization ability of the model; Data batching: Divide the data into multiple small batches, and input the data of each batch into the model for training at the same time; Data loading: Use multi-threaded or multi-process data loading mechanisms to speed up data reading and load data into GPU memory; In the data batching, the batch size is B, the total data volume is N, and the data volume of each batch is:

3. The model training method based on an algorithm of a specified GPU thread according to claim 1, characterized in that: The GPU thread management includes: Thread quantity control: Dynamically adjust the number of concurrent threads to ensure maximum computing efficiency while avoiding thread competition and resource waste; Thread scheduling strategy: By setting thread priorities, key computing tasks are prioritized and the resource usage of low-priority tasks is reduced; Thread Binding and Core Affinity: Bind threads to specific GPU cores to reduce thread switching and cross-core communication overhead; in, The number of threads is Nt, and the priority weight of the task is Wt. The resource ratio Rt obtained by each thread is expressed as:

4. The model training method based on an algorithm of a specified GPU thread according to claim 1, characterized in that: The memory optimization and data transmission include: Memory reuse: During training, the buffers in GPU memory are reused to reduce dynamic memory allocation and release operations and avoid fragmentation. Data pipeline and cache: Pre-fetch the next batch of data while the GPU is performing computing tasks and store it in the cache to achieve parallelization of computing and data loading; Compression and decompression: compress data before transmission to reduce transmission time, and decompress quickly on the GPU side; in, The data transmission time before compression is Td, and the compression ratio is C. Then the transmission time after optimization is ToptT:

5. The model training method based on an algorithm of a specified GPU thread according to claim 1, characterized in that: The parallel training algorithm comprises: Data parallelism: The training data set is divided into multiple subsets, which are trained independently on different GPUs. After each GPU calculates the gradient, it is aggregated and the model parameters are updated. Model parallelism: Divide different parts of the model into different GPUs for computing, which is suitable for training large models; Distributed training: Model training is performed on multiple machines, with multiple GPUs running on each machine, and gradient synchronization and parameter updates are performed through efficient network communication protocols.

6. The model training method based on an algorithm of a specified GPU thread according to claim 1, characterized in that: The model parallelism includes: Data parallelism: Let the training data set be D, containing N samples. Use k GPUs for parallel training and divide the data set into k subsets. The amount of data in each subset is: For each subset Di, the loss function of the jth iteration is: Where θ(j) is the model parameter of the jth iteration, L is the loss function, and f(x;θ) is the output of the model; Each GPU calculates the gradient of the corresponding subset: After all GPUs have calculated the gradients, they use the gradient synchronization mechanism to aggregate all gradients and update the model parameters: Where η is the learning rate; Model Parallelism: Assume that the model is divided into k parts and calculated on k GPUs respectively. For the i-th GPU, it is responsible for calculating the output of the i-th part of the model: Hi=f;(H1;0i) Among them, Hi is the output of the i-th layer, Hi-1 is the output of the previous layer, and θi is the parameter of the i-th layer; during the back propagation process, the calculation of the gradient of each part is as follows: at the same time, Passed to the previous layer via the chain rule: When updating parameters, after gradient aggregation, use synchronous update or asynchronous update: Distributed training: Suppose there are M machines, each with km GPUs, then the total number of GPUs is The size of the data subset on each machine is: Each subset is further divided into corresponding GPUs. Dm,i is the data subset processed by the i-th GPU on the m-th machine: The global parameter update formula is:

7. The model training method based on an algorithm of a specified GPU thread according to claim 1, characterized in that: The automatic performance adjustment includes: Adaptive learning rate adjustment: Dynamically adjust the learning rate based on the training error or loss function value to avoid overfitting or non-convergence of training; Load balancing and resource allocation: monitor the utilization of each GPU in real time and dynamically adjust the allocation of computing tasks to balance the load of all GPUs, thereby improving overall computing efficiency; Performance monitoring and feedback tuning: By real-time monitoring of key indicators such as CPU, GPU utilization, memory usage, and network bandwidth during training, training parameters and scheduling strategies are automatically adjusted based on feedback.

8. An electronic device, comprising a processor and a memory connected to the processor for storing instructions executable by the processor, characterized in that: The processor is used to execute a model training method based on an algorithm of a specified GPU thread as described in any one of claims 1-7 above.

9. A server, characterized in that: It includes at least one processor and a memory communicatively connected to the processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the processor so that the at least one processor executes a model training method based on an algorithm of a specified GPU thread as described in any one of claims 1-7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the model training method based on an algorithm of a specified GPU thread as described in any one of claims 1 to 7 is implemented.

Citation Information

Cited By

  • Operation cost estimation method and system based on machine learning

    CN119444290A

  • A multi-threaded intelligent furniture control method and system

    CN119758751A