A heterogeneous GPU allocation system and method for multiple deep learning tasks in a distributed environment

Abstract

The present invention belongs to the field of deep learning under artificial intelligence, and discloses a heterogeneous GPU allocation system and method for multiple deep learning tasks in a distributed environment. The system includes a GPU Profile module, a task information collection module, a GPU selection module, and a deep learning training module. A heterogeneous GPU allocation method for multiple deep learning tasks in a distributed environment according to the present invention can allocate GPUs with different computing capabilities to tasks with corresponding requirements, adapt tasks with complex model hierarchies and large amounts of batch data to nodes with the best performance GPUs and sufficient video memory for storage, speed up tasks that require longer deep learning training, and thus significantly improve the multi-task execution efficiency in a heterogeneous environment. Moreover, when multiple deep learning tasks are executed concurrently, multiple deep learning tasks can be completed faster as a whole, saving the time for programmers or users to wait for results.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning under artificial intelligence, and particularly relates to a heterogeneous GPU allocation system and method for multiple deep learning tasks in a distributed environment. Background Art

[0002] Nowadays, deep neural networks obtain very accurate models through training with large-scale data, which promotes the continuous application of deep neural networks in fields such as image classification, speech recognition, and driverless. These trends have led to the increasing complexity of deep neural network models and the continuous emergence of devices for accelerating deep neural network training, such as GPUs, FPGAs, TPUs, etc. Therefore, how to more efficiently utilize heterogeneous acceleration devices in a distributed environment has gradually become an important research focus.

[0003] Concurrent deep learning training of multiple tasks in a distributed heterogeneous GPU environment has gradually become a common phenomenon. One scenario is the collaborative training of multiple deep learning tasks in a distributed environment, and each task needs to complete a training index. Facing this scenario, how to design an efficient GPU allocation scheme to improve the overall execution efficiency of multiple deep learning training tasks has become an important research field.

[0004] Currently, the traditional GPU allocation method of deep learning training frameworks generally statically specifies GPU parameters when starting multiple tasks in a distributed environment, and uses the GPU selection parameters provided by the deep learning training framework to schedule tasks with different requirements to corresponding GPUs for deep learning training; the deep learning training framework also provides a method of using all available GPUs, which allows the batch data of each task to be evenly distributed to all GPUs for deep learning training. Since the GPUs with strong computing power quickly complete the evenly distributed small batches of data, this GPU allocation scheme causes the GPUs with strong computing power to be idle for a long time, and the resulting impact is that the utilization rate of the GPUs with strong computing power is not high.

[0005] Since the traditional GPU allocation scheme for multiple deep learning tasks in a distributed environment does not consider the characteristics and requirements of tasks and does not fully utilize the performance of heterogeneous GPUs to meet the concurrent operation of different deep learning training tasks, the overall execution efficiency of multiple deep learning tasks is not high.

[0006] Aiming at the deficiencies of the above traditional scheme, a scenario of multiple deep learning tasks in a distributed environment requires an effective GPU allocation method, and the problems in this scenario urgently need an effective solution. Summary of the Invention

[0007] The purpose of the present invention is to provide a heterogeneous GPU allocation system and method for multiple deep learning tasks in a distributed environment to solve the above technical problems.

[0008] To solve the above technical problems, the specific technical solutions of a heterogeneous GPU allocation system and method for multiple deep learning tasks in a distributed environment of the present invention are as follows:

[0009] A heterogeneous GPU allocation system for multiple deep learning tasks in a distributed environment includes a GPU Profile module, a task information collection module, a GPU selection module, and a deep learning training module;

[0010] The GPU Profile module: is responsible for detecting whether the machines in the heterogeneous environment contain GPUs, the performance of the GPUs, and the size of the GPU video memory;

[0011] The task information collection module: is responsible for collecting the training models, data batch sizes of each task, and the training time of the batch data of each task;

[0012] The GPU selection module: is responsible for target GPU selection, and distributing the task batch data from the memory cache space to the GPUs;

[0013] The deep learning training module: is responsible for applying the decision GPU information sent by the GPU selection module to this module, and obtaining the task model information and data batch size information, so as to perform the corresponding network layer deep learning training calculation on this GPU.

[0014] The present invention also discloses a heterogeneous GPU allocation method for multiple deep learning tasks in a distributed environment, including the following steps:

[0015] S1, Initialization of multiple deep learning training tasks;

[0016] S2, Cold start of multiple deep learning training tasks;

[0017] S3, Dynamic adjustment of the GPU scheme for multiple deep learning training tasks;

[0018] S4, Loop iteration of multiple deep learning training tasks.

[0019] Further, when initializing each deep learning training task in S1, the GPU Profile module collects the feature information of each GPU in the heterogeneous environment and records the parameter information brought by the task startup, and these information are the reference factors for the GPU selection module.

[0020] Further, when starting the first Epoch of the deep learning training of multiple tasks in S2, the GPU selection module gives a GPU allocation scheme for the cold start of the deep learning training of multiple tasks.

[0021] Further, S2 includes the following specific steps:

[0022] S21, sort all tasks according to the size of the deep learning training network layer of the task, using the model type of the task as the first-priority sorting factor;

[0023] S22, sort all tasks again according to the data batch size of the task deep learning training, using the data batch size of the task as the second-priority sorting factor;

[0024] S23, sort all GPUs according to the computing power of the GPU, using the computing power of the GPU as the first-priority sorting factor;

[0025] S24, sort all GPUs again according to the video memory size of the GPU, using the video memory size of the GPU as the second-priority sorting factor;

[0026] S25, then, according to the order sorted in steps S22 and S24, assign the first to the last GPU to the first to the last sorted task in turn according to this order, and record the one-to-one mapping information in the global GPU allocation table as the allocation scheme for cold start.

[0027] Further, in step S3, according to the cold start scheme, start the training of the first Epoch of multiple deep learning tasks. The task performance calculation module will calculate the training time of the batch data of each task on the corresponding GPU, record it in the global running time list, and pass it to the allocation algorithm of the GPU selection module to dynamically adjust the current GPU allocation and optimize the current GPU allocation until the best scheme.

[0028] Further, step S3 includes the following specific steps:

[0029] S31, each current task has already performed the training of the batch data of min-batch. Record the training running time of the current batch data of each task through the log, and according to the batch data time of multiple tasks, find the one with the largest time value, that is, obtain the longest running time of the current GPU allocation scheme, and record it as T cur ;

[0030] S32. In the multi-task start of deep learning training in the current GPU allocation scheme, when the training of the first batch of data for each task ends, calculate the training running time of the batch data for each task, that is, the information in the global running time list. Re-allocate the GPUs of the tasks with the longest and shortest running times recorded in the global running time list, so that the GPU with stronger computing power executes the task with the longest current running time, and the GPU with relatively weaker computing power executes the task with the shortest running time, that is, obtain a new GPU allocation scheme, and then calculate the running time of the next batch of deep learning training for each task, and then obtain the longest running time of the new GPU allocation scheme, recorded as T next , if the execution efficiency of the new scheme is better than the previous one, record the overall optimal performance as T best , and modify the new GPU allocation scheme in the global GPU allocation table;

[0031] S33. When the batch data of each task in the current multi-task has completed running and the running time is recorded in the global running time list, at this time, the GPU allocation algorithm will update the GPU allocation scheme and modify it in the global GPU allocation table. When the allocation scheme is for each task to start the next batch of data for deep learning training, it will automatically set parameters to specify that the task performs deep learning training on the corresponding GPU;

[0032] S34. In the first Epoch stage of each task, the allocation algorithm in the GPU selection module adjusts the scheme to the optimal execution efficiency by continuously taking the batch data of each task for training iteration. After the first Epoch training of the deep learning training of each current task is completed, select a multi-task execution efficiency optimal scheme as the final scheme for the next multiple rounds of deep learning training of each task.

[0033] Furthermore, in S4, when the first Epoch training of multiple deep learning tasks is completed, the deep learning training of each subsequent Epoch of the multi-task will distribute the batch data of each task to the corresponding GPU for deep learning training according to the best GPU allocation scheme generated in the first Epoch. If the training rounds of all tasks have been completed, the deep learning training task ends.

[0034] A heterogeneous GPU allocation system and method for multiple deep learning tasks in a distributed environment according to the present invention has the following advantages: The present invention proposes a heterogeneous GPU allocation system and method for multiple deep learning tasks in a distributed environment, which can allocate GPUs with different computing capabilities to tasks corresponding to the requirements, adapt tasks with complex model hierarchies and large batch data volumes to nodes with the best performance GPUs and sufficient video memory for storage, and speed up tasks that require longer deep learning training times, thereby significantly improving the multi-task execution efficiency in a heterogeneous environment; moreover, when multiple deep learning tasks are executed concurrently, multiple deep learning can be completed faster as a whole, saving the time for programmers or users to wait for the results. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a module diagram of the system logic architecture using the method of the present invention.

[0036] Figure 2 It is a schematic diagram of the data flow of multi-task deep learning in the present invention.

[0037] Figure 3 It is a flowchart of the GPU allocation algorithm in the present invention.

[0038] Figure 4 It is an example diagram of GPU allocation during cold start in the present invention.

[0039] Figure 5 It is a schematic diagram of the GPU allocation scheme in the present invention.

[0040] Figure 6 It is a performance verification diagram of an example of the GPU selection scheme. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0041] In order to better understand the purpose, structure and function of the present invention, the following further describes in detail a heterogeneous GPU allocation system and method for multiple deep learning tasks in a distributed environment according to the present invention with reference to the accompanying drawings.

[0042] As Figure 1 shown, a heterogeneous GPU allocation system for multiple deep learning tasks in a distributed environment according to the present invention includes a GPU Profile module, a task information collection module, a GPU selection module, and a deep learning training module;

[0043] The GPU Profile module: responsible for detecting whether the machine in the heterogeneous environment contains a GPU, the strength of the GPU performance, and the size of the GPU video memory;

[0044] The task information collection module: responsible for collecting the training model, data batch size, and training time of each batch of data for each task;

[0045] GPU Selection Module: Responsible for target GPU selection and distributing task batch data from the memory cache space to the GPU;

[0046] Deep Learning Training Module: Responsible for applying the decision GPU information sent by the GPU Selection Module to this module, and obtaining task model information and data batch size information, so as to perform corresponding network layer deep learning training calculations on this GPU.

[0047] A heterogeneous GPU allocation method for multiple deep learning tasks in a distributed environment according to the present invention includes the following steps:

[0048] S1, Initialization of multiple deep learning training tasks; When each deep learning training task is initialized, the GPUProfile module collects the feature information of each GPU in the heterogeneous environment and records the parameter information brought by the task startup. These information are the reference factors for the GPU Selection Module;

[0049] S2, Cold start of multiple deep learning training tasks; When the first Epoch of the deep learning training of multiple tasks starts, the GPU Selection Module gives a GPU allocation plan for the cold start of multiple task deep learning training;

[0050] S21, Sort all tasks according to the model type of the task as the first priority sorting factor according to the size of the deep learning training network layer of the task;

[0051] S22, Sort all tasks again according to the size of the batch data of the task as the second priority sorting factor according to the data batch size of the task deep learning training;

[0052] S23, Sort all GPUs according to the computing power of the GPU as the first priority sorting factor according to the strength of the GPU computing power;

[0053] S24, Sort all GPUs again according to the video memory size of the GPU as the second priority sorting factor according to the video memory size of the GPU;

[0054] S25, Then according to the order sorted in steps S22 and S24, assign the GPUs to the sorted first to last tasks in turn from the first to the last GPU, and the one-to-one mapping information is recorded in the global GPU allocation table as the allocation plan for cold start.

[0055] S3. Dynamic adjustment of the GPU solution for multiple deep learning training tasks; according to the cold start solution, start the first Epoch training of multiple deep learning tasks. The task performance calculation module will calculate the training time of the batch data of each task on the corresponding GPU and record it in the global running time list, and pass it to the allocation algorithm of the GPU selection module to dynamically adjust the current GPU allocation and optimize the current GPU allocation until the best solution. The steps are as follows:

[0056] S31. Each current task has already performed the training of the batch data of one min-batch. Record the training running time of the current batch data of each task through the log, and based on the batch data time of multiple tasks, find the maximum value among them, that is, obtain the longest running time of the current GPU allocation solution, and record it as T cur ;

[0057] S32. Start the deep learning training of multiple tasks in the current GPU allocation solution. When the training of the first batch of data of each task ends, calculate the training running time of the batch data of each task, that is, the information in the global running time list. Re-allocate the GPUs of the tasks with the longest and shortest running times recorded in the global running time list, so that the GPU with stronger computing power executes the task with the longest current running time, and the GPU with relatively weaker computing power executes the task with the shortest running time, that is, obtain a new GPU allocation solution, and then find the longest running time of the new GPU allocation solution during the next batch deep learning training of each task, and record it as T next， If the execution efficiency of the new solution is better than the previous one, record the overall optimal performance as T best and modify the new GPU allocation solution in the global GPU allocation table;

[0058] S33. The batch data of each task in the current multiple tasks has completed running, and the running time is recorded in the global running time list. At this time, the GPU allocation algorithm will update the GPU allocation solution and modify it in the global GPU allocation table. The allocation solution will automatically set parameters to specify the task to perform deep learning training on the corresponding GPU when each task has not started the deep learning training of the next batch of data;

[0059] S34. In the first Epoch stage of each task, the allocation algorithm in the GPU selection module adjusts the solution through continuous training iterations of taking the batch data of each task until the execution efficiency reaches the optimal. After the first Epoch training of the deep learning training of each current task is completed, the algorithm of the present invention can select a solution with the optimal execution efficiency of multiple tasks as the final solution for the next multiple rounds of deep learning training of each task;

[0060] S4. Multiple deep learning training tasks iterate; when the training of the first Epoch of multiple deep learning tasks is completed, the deep learning training of each subsequent Epoch of multiple tasks will distribute the data of each task batch to the corresponding GPU for deep learning training according to the optimal GPU allocation scheme generated in the first Epoch. If the training rounds of all tasks are completed, the deep learning training task ends.

[0061] Embodiment:

[0062] The present invention designs a heterogeneous GPU allocation method for multiple deep learning tasks in a distributed environment according to factors such as the current heterogeneous GPU configuration, whether the video memory capacity of the GPU can hold the batch data volume of multiple tasks at the same time, and the execution efficiency of each task's deep learning training. The method includes the following steps:

[0063] S1. Initialize multiple deep learning training tasks; as Figure 2 shows the data stream of multiple tasks performing deep learning training in a heterogeneous environment. The heterogeneity shown in the present invention only lies in the differences in GPUs. The underlying storage of the original data is transmitted to the DRAM cache in the heterogeneous environment, and the time required for each task in this process is approximately the same. The present invention needs to optimize how the batch data for the memory cache is better allocated to the corresponding GPUs. When initializing multiple-task deep learning, the GPU Profile module collects the heterogeneous GPU feature information, denoted as GpuInfoMap<gpuId, [computerCapablity, memSize]>, where gpuId represents the unique number of the GPU in the environment, computerCapablity represents the numerical value of the GPU computing power, and memSize represents the numerical value of the GPU video memory. The information in GpuInfoMap is used as a reference factor for the GPU selection module. The task starts with its own parameter information, denoted as JobInfoMap<jobId, [modelType, batchSize]>, where jobId represents the unique number of the deep learning training task, modelType represents the model type of the deep learning training, and batchSize represents the size of the batch data for the deep learning training, which is used as a reference factor for the GPU selection module; the data structures recording these static information are as Figure 4 shown, and these data structures are recorded in the global memory throughout the operation of multiple tasks.

[0064] S11. The computing capabilities of each GPU in the heterogeneous environment can be read through the cluster configuration information file and converted into an integer value to represent the computing performance of the GPU through the computing power numerical table provided by the Nvidia official website;

[0065] S12, The operating parameters for multi-task deep learning training can be pre-read in the startup command script of the multi-task and recorded in the JobInfoMap data structure in the global memory.

[0066] S2, Cold start of multi deep learning training tasks; when the first Epoch of the multi-task deep learning training starts, the GPU selection module gives a GPU allocation plan for the cold start of multi-task deep learning training. An example of this allocation plan is as Figure 4 shown.

[0067] Step S2 can adopt sorting rules with different priorities of multiple factors to obtain the GPU allocation plan during cold start, and implement it according to the following steps:

[0068] S21, Using the model type of the task as the first sorting factor, perform a priority sorting from large to small according to the number of network layers of the task's deep learning training, and sort all tasks recorded in JobInfoMap;

[0069] S22, Using the size of the batch data of the task as the second sorting factor, perform a priority sorting from large to small according to the data batches of the task's deep learning training, and sort all tasks recorded in JobInfoMap again;

[0070] S23, Using the computing power of the GPU as the first sorting factor, perform a priority sorting from strong to weak according to the GPU computing power, and sort all GPUs recorded in GpuInfoMap;

[0071] S24, Using the video memory size of the GPU as the second sorting factor, perform a priority sorting from large to small according to the GPU video memory, and sort all GPUs recorded in GpuInfoMap again;

[0072] S25, Then according to the order sorted in steps S22 and S24, assign the first to the last GPU in GpuInfoMap to the first to the last task in JobInfoMap in this order one by one. The mapped information is recorded in the global GPU allocation table, denoted as GpuAllocList, as this cold start allocation plan.

[0073] S3, Dynamic adjustment of the multi deep learning training task GPU plan; when the first Epoch stage of the multi-task deep learning training is executed, the task performance calculation module will calculate the training time of the batch data of each task on the corresponding GPU, record it in the global running time list, denoted as T[jobId], and pass it to the allocation algorithm of the GPU selection module to dynamically adjust the current GPU allocation and optimize the current GPU allocation until the best plan. The algorithm implementation steps of the present invention are as Figure 3As shown below, the algorithm steps are further explained as follows:

[0074] S31. Obtain the multitask running time of the current GPU allocation scheme, and find the maximum time value among them by using the multitask batch data time recorded in T[jobId], that is, obtain the longest running time of the current scheme, and record it as T cur ; As shown in the following formula:

[0075] T[jobId] = T(jobId_batch)

[0076] T cur = Max(T[jobId])

[0077] S32. Start deep learning training for multitasks in the current GPU allocation scheme. When the training of the first batch of data for each task ends, calculate the training running time of the batch data for each task, that is, the information in T[jobId]. Reallocate the GPUs of the tasks with the longest and shortest running times recorded in T[jobId], so that the GPU with stronger computing power executes the task with the longest current running time, and the GPU with relatively weaker computing power executes the task with the shortest running time, that is, obtain a new GPU allocation scheme. As Figure 5 shown, reselect the most suitable GPUs for the two tasks in the first scheme, and the second scheme is better than the first scheme. Record the longest running time obtained from the deep learning training of batch data for multiple tasks in the new scheme, and record it as T next , if the execution efficiency of the new scheme is better than the previous scheme, record the overall optimal performance as T best , and record the new GPU allocation scheme in GpuAllocList. As shown in the following formula:

[0078] T best = min( T cur， T next )

[0079] if T best is minest, GpuAllocList = [...,(jobId, newGpuId),...]

[0080] S33. For each task in the current multitasking, the batch data of each task has completed running, and the running time is recorded in T[jobId]. At this moment, the GPU allocation algorithm will update the GPU allocation plan and modify it in the global GPU allocation table, that is, update GpuAllocList. When the allocation plan is used for the deep learning training of the next batch of data for each task that has not been started, the parameters will be automatically set to specify that the task performs deep learning training on the corresponding GPU. If the multitasking at the current moment is already running the deep learning training of a batch of data, at this moment, the multitasking needs to complete the deep learning training of the current batch of data according to the previous GPU allocation until the next batch of data for each task is ready, and then a new GPU allocation plan is adopted to execute the deep learning training;

[0081] S34. In the first Epoch stage of each task, the allocation algorithm in the GPU selection module adjusts the plan by continuously taking the batch data of each task for training iteration until the execution efficiency of the multitasking reaches the optimal. The finally selected plan with the optimal execution efficiency of the multitasking is used as the final plan for the next multiple rounds of deep learning training of each task. As Figure 6 shown in an example, the optimal plan provided by the design method and system of the present invention is Plan 2. By executing the entire deep learning training process of the multitasking through Plan 2, the execution efficiency of the multitasking is improved compared with other plans;

[0082] S4. The multiple deep learning training tasks are iterated in a loop; when the first Epoch training of the multiple deep learning tasks is completed, the deep learning training of each subsequent Epoch of the multitasking will distribute the batch data of each task to the corresponding GPU for deep learning training according to the optimal GPU allocation plan generated in the first Epoch. If the training rounds of all tasks are completed, the deep learning training task ends.

[0083] It can be understood that the present invention is described through some embodiments. Those skilled in the art know that without departing from the spirit and scope of the present invention, various changes or equivalent replacements can be made to these features and embodiments. In addition, under the teaching of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application belong to the scope protected by the present invention.

Claims

1. A heterogeneous GPU allocation method for multiple deep learning tasks in a distributed environment, characterized in that The steps include: S1, initialization of multiple deep learning training tasks; S2, cold start of multiple deep learning training tasks; S21, using the model type of the task as the first priority sorting factor, sorting all tasks by the size of the deep learning training network layer of the task; S22, using the batch data size of the task as the second priority sorting factor, sorting all tasks according to the data batch size of the task deep learning training; S23, using the computing capability of the GPU as the first priority sorting factor, sorting all GPUs according to the computing capability of the GPU; S24, using the GPU memory size as the second priority sorting factor, sorting all GPUs according to the GPU memory size; S25, then according to the order arranged in steps S22 and S24, the first to the last GPU are allocated to the first to the last tasks in the order, and the one-to-one mapping information is recorded in the global GPU allocation table as the cold start allocation plan; S3, dynamic adjustment of GPU solutions for multiple deep learning training tasks; S31. Each task has now performed one min-batch of data training. Record the training running time of the current batch of data for each task through the log, and based on the batch data times of multiple tasks, find the maximum time value among them, which is the longest running time of the current GPU allocation scheme, recorded as T cur ; S32. In the multi-task startup deep learning training in the current GPU allocation scheme, when the training of the first batch of data for each task ends, calculate the training running time of the batch data for each task, that is, the information in the global running time list. Reallocate the GPUs of the tasks with the longest and shortest running times recorded in the global running time list, so that the GPU with stronger computing power executes the task with the longest current running time, and the GPU with relatively weaker computing power executes the task with the shortest running time, that is, obtain a new GPU allocation scheme, and calculate the running time of the next batch deep learning training for each task, and then obtain the longest running time of the new GPU allocation scheme, which is recorded as T next , if the execution efficiency of the new scheme is better than the previous one, record the overall optimal performance as T best , and modify the new GPU allocation scheme in the global GPU allocation table; S33, the batch data of each task of the current multi-task has been completed, and the running time is recorded in the global running time list. At this moment, the GPU allocation algorithm will update the GPU allocation plan and modify it in the global GPU allocation table. The allocation plan will automatically set parameters to specify the task to perform deep learning training on the corresponding GPU when each task has not started to perform deep learning training for the next batch of data; S34, in the first epoch stage of each task, the allocation algorithm in the GPU selection module continuously takes batch data of each task for training iteration to adjust the plan to achieve the best execution efficiency. After the first epoch of deep learning training of each task is completed, a plan with the best multi-task execution efficiency is selected as the final plan for the next multiple rounds of deep learning training of each task; S4, multiple deep learning training tasks are iterated cyclically.

2. The heterogeneous GPU allocation method for multiple deep learning tasks in a distributed environment according to claim 1, wherein When each deep learning training task is initialized, the GPU Profile module in S1 collects feature information of each GPU in a heterogeneous environment and records the parameter information of the task startup. This information is a reference factor for the GPU selection module.

3. The heterogeneous GPU allocation method for multiple deep learning tasks in a distributed environment according to claim 1, wherein When the first Epoch of multi-task deep learning training is started in S2, the GPU selection module provides a GPU allocation plan for the cold start of multi-task deep learning training.

4. The heterogeneous GPU allocation method for multiple deep learning tasks in a distributed environment according to claim 1, wherein The S3 starts the first Epoch training of multiple deep learning tasks according to the cold start solution. The task performance calculation module calculates the training time of the batch data of each task on the corresponding GPU, records it in the global running time list, and passes it to the allocation algorithm of the GPU selection module to dynamically adjust the current GPU allocation and optimize the current GPU allocation to the best solution.

5. The heterogeneous GPU allocation method for multiple deep learning tasks in a distributed environment according to claim 1, wherein When the first Epoch of the multi-deep learning tasks is completed, for each subsequent Epoch of the multi-tasks, the deep learning training will distribute the data batches of each task to the corresponding GPUs for deep learning training according to the optimal GPU allocation scheme generated in the first Epoch. If the training rounds of all tasks are completed, the deep learning training task ends.

6. A system for a heterogeneous GPU allocation method for multiple deep learning tasks in a distributed environment as described in claim 1, characterized in that, The system includes a GPU Profile module, a task information collection module, a GPU selection module, and a deep learning training module; The GPU Profile module: responsible for detecting whether the machine in the heterogeneous environment has a GPU, the strength of the GPU performance, and the size of the GPU video memory; The task information collection module: responsible for collecting the training model of each task, the data batch size, and the training time of the batch data of each task; The GPU selection module: responsible for target GPU selection and distributing the task batch data from the memory cache space to the GPU; The deep learning training module: responsible for applying the decision GPU information sent by the GPU selection module to this module, and obtaining the task model information and data batch size information, so as to perform the corresponding network-level deep learning training calculation on this GPU.

Images