A graphics processor over-tasking method and device

Abstract

The application provides a graphics processor excess task allocation method and device, wherein the method comprises the following steps: dividing task stages for a plurality of to-be-allocated tasks according to resource consumption data of each to-be-allocated task, and generating an interleaved packaging scheme for at least two to-be-allocated tasks based on the task stages, so as to optimize GPU sharing in the time dimension; the algorithm identifies tasks with complementary resource requirements and binds them to be packaged, so as to relieve resource competition in the time dimension; based on the interleaved packaging scheme, GPU is allocated to the to-be-allocated tasks in a blocked scheduling mode combined with a non-blocked scheduling mode according to the arrival time of the to-be-allocated tasks, so as to avoid head blocking and straggler problems in the space dimension; wherein at least one processor is allocated to-be-allocated tasks that exceed the total sum of resource capacities of the processor, and in combination with a task alternation execution mechanism, GPU utilization is improved by allowing tasks to temporarily overlap their resource requirements.

Description

Technical Field

[0001] This invention belongs to the field of computer technology, specifically relating to a method and apparatus for allocating excess tasks to a graphics processor. Background Technology

[0002] As deep learning (DL) models continue to grow in complexity and scale, the demand for computing resources, especially GPUs (Graphics Processing Units), is constantly increasing. Therefore, training in cloud environments is becoming increasingly common. Users typically submit deep learning training (DLT) jobs to the cloud, where cloud services allocate resources based on GPU availability and execute them.

[0003] During DLT execution, redundant resource allocation and dynamic resource usage result in low GPU utilization, leading to longer Job Completion Time (JCT). For DLT job scheduling, related technologies typically require that the total resource requirements of DLT jobs on the GPU cannot exceed the GPU's capacity. However, since it is difficult to perfectly match multiple DLT jobs to full GPU load, this limitation often exacerbates resource waste. Summary of the Invention

[0004] To address the aforementioned technical issues, this invention provides a method and apparatus for allocating excess tasks to a graphics processor.

[0005] This invention provides a method for allocating excess tasks to a graphics processor, comprising: acquiring a plurality of tasks to be allocated and resource consumption data of each task to be allocated; dividing the plurality of tasks to be allocated into task stages according to the resource consumption data of each task to be allocated, and generating an interleaved packing scheme for at least two tasks to be allocated based on the task stages, wherein the interleaved packing scheme includes enabling one processor to process at least two tasks to be allocated simultaneously; and allocating processors to the plurality of tasks to be allocated based on the interleaved packing scheme and according to the arrival time of the tasks to be allocated, using blocking scheduling and non-blocking scheduling, wherein at least one processor is allocated tasks exceeding the total resource capacity of that processor.

[0006] Optionally, the resource consumption data includes the resource consumption of the task to be assigned at different times; dividing the task to be assigned into task stages based on the resource consumption data includes: dividing the task stages of the task to be assigned into a cross-training cycle stage and a training cycle stage based on the resource consumption of the task to be assigned at different times, wherein, for at least one resource, the resource consumption of the training cycle stage is greater than the resource consumption of the cross-training cycle stage; the interleaved packaging scheme includes: enabling one of the processors to simultaneously process one task to be assigned in the cross-training cycle stage and another task to be assigned in the training cycle stage.

[0007] Optionally, the above-mentioned generation of an interleaved packaging scheme for at least two tasks to be assigned based on the above-mentioned task stage includes: for at least two tasks to be assigned, calculating the resource complementarity value between each pair of tasks to be assigned, and adding the packaging scheme of the two tasks to be assigned with the highest resource complementarity value to the above-mentioned interleaved packaging scheme; the formula for calculating the resource complementarity value includes: ; ;in, Tasks to be assigned Tasks to be assigned The complementary value of resources Used to indicate tasks to be assigned. Multiple calculation indicators, Used to indicate tasks to be assigned. Multiple calculation indicators, The function is used to calculate complementary values. Indicates taking and The maximum value in the list, the task to be assigned Tasks to be assigned The various computational metrics include the number of training cycles, average CPU utilization across the aforementioned training cycle phases, maximum memory usage, average GPU utilization, maximum GPU memory usage, and time required, as well as the average CPU utilization, maximum memory usage, average GPU utilization, maximum GPU memory usage, and duration of each phase within the aforementioned training cycle.

[0008] Optionally, before generating an interleaved packaging scheme for at least two tasks to be assigned based on the task stages, the method further includes: calculating a matching factor based on the duration of each task stage of each task to be assigned; generating the interleaved packaging scheme for the task to be assigned in response to the matching factor of the task to be assigned being greater than or equal to a preset matching factor threshold; or, generating a separate processor allocation scheme for the task to be assigned if the matching factor of the task to be assigned is less than the matching factor threshold; the formula for calculating the matching factor includes: ;in, Tasks to be assigned The above matching factors, Tasks to be assigned The duration of the aforementioned cross-training cycle phases, Tasks to be assigned The duration of each phase within the aforementioned training cycle.

[0009] Optionally, based on the arrival time of the tasks to be assigned, processors are allocated to multiple tasks to be assigned using blocking scheduling and non-blocking scheduling methods, including: classifying at least two tasks to be assigned into different time buckets based on their arrival time; sorting the different time buckets from earliest to latest according to their arrival time, and sorting the tasks to be assigned within each time bucket from highest to lowest priority; sequentially allocating processors to each sorted task to be assigned, wherein, in response to a task to be assigned that is ranked higher than the others in a time bucket and cannot be assigned a processor, the task to be assigned is skipped; and, in response to a situation where all tasks to be assigned in a time bucket ranked higher than the others cannot be assigned a processor, the allocation is paused until all tasks to be assigned in that time bucket are assigned processors.

[0010] Optionally, the multiple processors are distributed across multiple nodes, with each node including at least one processor. Before allocating processors to the multiple tasks to be assigned using blocking and non-blocking scheduling methods, the method further includes: selecting a subset of processors as available processors based on the remaining resources of each processor; sorting the multiple nodes from highest to lowest based on the number of available processors, and then sorting the nodes with the same number of available processors from highest to lowest based on the total remaining resources of the available processors within the node; within each node, sorting the multiple available processors from highest to lowest based on their remaining resources, to obtain a sorting result for all available processors in all nodes; the allocation of processors to the multiple tasks to be assigned using blocking and non-blocking scheduling methods includes: allocating available processors to the multiple tasks to be assigned according to the sorting result using blocking and non-blocking scheduling methods.

[0011] Optionally, the above method further includes: using the processor to simultaneously process one task to be assigned that is in the cross-training cycle stage and another task to be assigned that is in the training cycle stage, wherein the processor includes a stage lock, the stage lock being used to prevent the processor from simultaneously processing two or more tasks to be assigned that are in the same task stage.

[0012] Based on the same inventive concept, the present invention also provides a graphics processor excess task allocation device, comprising: a task analysis module, configured to acquire multiple tasks to be allocated and resource consumption data of each of the tasks to be allocated; a task scheduling module, configured to divide the multiple tasks to be allocated into task stages according to the resource consumption data of each task to be allocated, and generate an interleaved packing scheme for at least two tasks to be allocated based on the task stages, wherein the interleaved packing scheme includes enabling one processor to process at least two tasks to be allocated simultaneously; the task scheduling module is further configured to allocate processors to the multiple tasks to be allocated based on the interleaved packing scheme and according to the arrival time of the tasks to be allocated, through blocking scheduling and non-blocking scheduling, wherein at least one processor is allocated tasks exceeding the total resource capacity of the processor.

[0013] Based on the same inventive concept, the present invention also provides an electronic device, comprising: a processor; a memory for storing executable instructions; wherein the processor is configured to read the executable instructions from the memory and execute the executable instructions to implement any of the methods described above.

[0014] Based on the same inventive concept, the present invention also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the processor causes the processor to implement any of the methods described above.

[0015] The beneficial effects of this invention are as follows: First, it provides a novel complementary matching algorithm for job packaging to optimize GPU sharing in the time dimension. This algorithm identifies tasks with complementary resource requirements and packages and binds them to alleviate resource contention in the time dimension. Second, it introduces a novel multi-GPU orchestration algorithm. This algorithm dynamically allocates GPUs to tasks based on their arrival times, combining blocking and non-blocking scheduling methods to avoid head-blocking and lagging issues in the spatial dimension. Simultaneously, this invention allows for over-allocation of tasks to processors. Combined with the aforementioned task alternation execution mechanism, it improves GPU utilization by allowing tasks to temporarily overlap their resource requirements. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating a method for allocating excess tasks to a graphics processor, provided in an embodiment of the present invention;

[0019] Figure 2 This is a schematic diagram illustrating the resource requirements of the task to be assigned at different stages in an embodiment of the present invention;

[0020] Figure 3 This is a schematic diagram of the basic oversubscription strategy in related technologies;

[0021] Figure 4 A schematic diagram of an over-allocation strategy based on an interleaved packing scheme provided in an embodiment of the present invention;

[0022] Figure 5 This is a schematic diagram illustrating how a processor handles interleaved packaging tasks, as provided in an embodiment of the present invention.

[0023] Figure 6 This is a schematic diagram illustrating another method of using a processor to process interleaved packaging tasks, as provided in an embodiment of the present invention.

[0024] Figure 7 This is a schematic diagram of a multi-GPU orchestration method provided in an embodiment of the present invention;

[0025] Figure 8 This is a schematic diagram illustrating the results of CPU selection based on a typical greedy strategy in related technologies.

[0026] Figure 9 This is a schematic diagram showing the CPU selection result based on the multi-GPU orchestration method provided in this embodiment of the invention;

[0027] Figure 10 This is a schematic diagram of the HyperWeave scheduler architecture provided in an embodiment of the present invention;

[0028] Figure 11 This is a schematic diagram showing the results of a simulation comparison experiment between an embodiment of the present invention and related technologies.

[0029] Figure 12 A schematic diagram showing the results of a simulation comparison experiment between another embodiment of the present invention and related technologies;

[0030] Figure 13 A schematic diagram showing the results of a simulation comparison experiment between another embodiment of the present invention and related technologies;

[0031] Figure 14 A schematic diagram showing the results of a simulation comparison experiment between another embodiment of the present invention and related technologies;

[0032] Figure 15 A schematic diagram showing the results of a simulation comparison experiment between another embodiment of the present invention and related technologies;

[0033] Figure 16 A schematic diagram showing the results of a simulation comparison experiment between another embodiment of the present invention and related technologies;

[0034] Figure 17 A schematic diagram showing the results of a simulation comparison experiment between another embodiment of the present invention and related technologies;

[0035] Figure 18 This is a schematic diagram of the structure of a graphics processor overload allocation device provided in an embodiment of the present invention;

[0036] Figure 19 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0037] To better understand the above-mentioned objectives, features, and advantages of the embodiments of the present invention, the solutions of the embodiments of the present invention will be further described below. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other.

[0038] Many specific details are set forth in the following description in order to provide a full understanding of the embodiments of the present invention, but the embodiments of the present invention may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some embodiments of the embodiments of the present invention, and not all embodiments.

[0039] This invention tracks GPU usage in related technologies and finds that 81.75% of DLT tasks run on a single GPU, highlighting the critical importance of efficient GPU sharing for cluster performance. While improved GPU performance allows a single GPU to accommodate more DLT jobs, the conservative sharing strategy employed in related technologies results in low GPU utilization. GPU tracking analysis of these technologies shows that 80% of requests have GPU utilization below 20%. Low utilization increases costs and prolongs job completion times; therefore, improving utilization and reducing JCT (Job Completion Time) are crucial.

[0040] There are two main reasons for the low GPU utilization in related technologies: First, there is redundancy in resource allocation. Users typically plan more resources than they actually use, with about 88.78% of jobs using less than 50% of the requested GPU resources. The configuration files of DLT jobs may lead to redundant GPU demand, and the actual utilization rate is far lower than the value shown in the configuration files. Second, DLT jobs exhibit dynamic usage patterns at different training stages, resulting in fluctuations in resource demand. The GPU idle time is significant for different configurations and models, further reducing the overall utilization rate.

[0041] GPU sharing technologies in related fields can be divided into two main categories: The first category is to divide the GPU into micro-instances before job deployment, but this may lead to resource fragmentation, and the optimization goal is to minimize fragmentation; the second category is to package multiple jobs into one GPU, aiming to maximize resource utilization, but this is limited by the resource allocation restrictions of the platform, such as the restrictions on GPU allocation by the Kubernetes scheduling platform, which leads to resource waste.

[0042] In view of this, embodiments of the present invention provide a method for allocating excess tasks to a graphics processor, such as... Figure 1 As shown, it includes:

[0043] S1. Obtain multiple tasks to be assigned (i.e., DLT) and the resource consumption data of each task to be assigned.

[0044] S2. Divide multiple tasks into task stages based on the resource consumption data of each task to be assigned, and generate an interleaved packaging scheme for at least two tasks to be assigned based on the task stages. The interleaved packaging scheme includes enabling one processor to process at least two tasks to be assigned simultaneously.

[0045] Specifically, since the resource consumption of a task varies at different stages, packaging tasks based on their stages allows the processor to process tasks at off-peak times, thereby improving efficiency.

[0046] S3. Based on the staggered packaging scheme, and according to the arrival time of the tasks to be assigned, processors (i.e. GPUs) are allocated to multiple tasks to be assigned through blocking scheduling and non-blocking scheduling. At least one processor is allocated more tasks than the total resource capacity of that processor.

[0047] Specifically, blocking scheduling means that for a given task, the allocation of a processor will not be preempted by other tasks, thus preventing the task from being starved. Conversely, non-blocking scheduling prioritizes allocating processors to other tasks when a task cannot be allocated a processor for a short period, thereby avoiding head-of-line blocking (HOL) problems. The method described in this embodiment combines the advantages of both scheduling methods by employing different scheduling approaches under different circumstances.

[0048] Over-allocation occurs when at least one processor is assigned more tasks than its total resource capacity. This is achieved by merging the total GPU requirements of multiple DLT jobs to exceed the target GPU capacity, and then running them on the target GPU. In practice, over-allocation can be implemented by pre-occupying processor resources, theoretically achieving 100% utilization of GPU resources.

[0049] This invention provides a GPU overload task allocation scheduler, hereinafter referred to as HyperWeave, for executing the aforementioned GPU overload task allocation method. First, this invention provides a novel complementary matching algorithm (i.e., S2 above) for job packaging to optimize GPU sharing in the time dimension. This algorithm identifies tasks with complementary resource requirements and packages and binds them to alleviate resource contention in the time dimension. Second, this invention introduces a novel multi-GPU orchestration algorithm (i.e., S3 above). This algorithm dynamically allocates GPUs to tasks based on their arrival times, combining blocking and non-blocking scheduling methods to avoid head-blocking and lagging issues in the spatial dimension. Simultaneously, this invention allows for overload task allocation to processors, meaning that at least one processor is allocated more tasks than its total resource capacity. Combined with the aforementioned task alternation execution mechanism, GPU utilization is improved by allowing tasks to temporarily overlap their resource requirements. In related technologies, overload allocation is essentially a free competition for resources between tasks. Severe resource contention leads to increased job completion time. This invention avoids head-blocking problems and also prevents tasks from starving through the aforementioned multi-GPU orchestration algorithm.

[0050] Specifically, we define the workload factor ω as the sum of GPU requirements (or utilization) for all tasks on a single GPU. When the workload factor does not exceed 100% (i.e., ω≤100%), the allocation strategy is called low commitment; when the workload factor exceeds 100% (i.e., ω>100%), over-allocation occurs. The two allocation strategies exhibit different characteristics in optimizing resource utilization.

[0051] Low-commitment strategies involve imposing restrictions on resource usage for each task. This approach ensures the viability of all tasks by guaranteeing a minimum resource level for each task, preventing any single task from consuming all available GPU resources. Furthermore, due to the predictability of resource allocation, this method also delivers predictable performance for the GPU. While these restrictions are crucial for responsiveness and reliability, restricted resource sharing often leads to inefficient resource utilization; fixed resource allocation may prevent some tasks from fully utilizing the GPU, resulting in resource waste and longer execution times.

[0052] The over-allocation strategy allows multiple tasks to freely compete for GPU resources without mandatory resource constraints, resulting in several benefits. First, this dynamic resource utilization enables tasks to adjust their resource requests in real time according to their computational needs, thus making more efficient use of available GPU cores and memory. This preventative mechanism can significantly reduce idle resources, thereby reducing task execution time. Furthermore, when tasks flexibly request resources, GPU parallel processing can improve overall system throughput, especially when different models have different computational loads.

[0053] However, the intense resource competition caused by over-allocation can lead to insufficient resources for each task, thereby increasing task completion time. Specifically, simultaneous use of GPU memory increases task fragmentation and reduces available GPU memory, forcing the GPU into low-workspace, slow algorithms. This invention addresses these problems by combining the aforementioned complementary matching algorithm and multi-GPU orchestration algorithm, retaining the advantages of the over-allocation strategy in improving GPU resource utilization while resolving the issues arising from it.

[0054] Specifically, in some embodiments, the resource consumption data includes the resource consumption of the task to be assigned at different times. The step in S2 above, which divides the task to be assigned into task stages based on the resource consumption data, includes:

[0055] Based on the resource consumption of the tasks to be assigned at different times, the task phases of the tasks to be assigned are divided into cross-training cycle phases and intra-training cycle phases. For at least one resource, the resource consumption in the intra-training cycle phase is greater than the resource consumption in the cross-training cycle phase.

[0056] The staggered packing scheme in S2 above includes:

[0057] This allows a processor to simultaneously handle one task to be assigned that is in a phase across training cycles and another task to be assigned that is in a phase within a training cycle.

[0058] The method provided in the above embodiments reduces the peak resource usage in the time dimension to avoid excessive resource competition, specifically by overlapping the peaks and troughs of resource usage for two tasks to be assigned. To more effectively stagger the peaks and troughs of resource usage for two tasks to be assigned, this embodiment divides the training process into an initial stage, a cross-training-cycle stage (hereinafter referred to as the cross-cycle stage), and a training-cycle-intra-stage stage (hereinafter referred to as the intra-cycle stage). The training cycle refers to the training cycle of DLT, where one training cycle represents the process of a deep learning model completing one learning and parameter update on the entire training dataset. A training cycle is relatively long, and dividing the process into stages according to the training cycle can further improve marginal returns. The DLT task in the cross-cycle stage includes data loading and checkpointing tasks, while the intra-cycle stage includes forward, backward, and gradient synchronization tasks. Different stages have different resource requirements, leading to fluctuations in resource usage, such as... Figure 2 As shown, Figure 2 The red waveform in the image represents the resource consumption of a certain DLT at different stages.

[0059] The above embodiments of the present invention allow some tasks to utilize GPU resources released by other tasks by interleaving two tasks to be assigned in cross-cycle and intra-cycle phases, thereby improving GPU utilization and reducing peak fluctuations in GPU utilization.

[0060] Specifically, the related technologies do not employ an interleaved packaging scheme, but only an over-allocation strategy for GPU usage, as illustrated in the diagram below. Figure 3 As shown, in GPU multiprocessor sharing mode, the basic oversubscription strategy causes four tasks (task 1, task 2, task 3, and task 4, with resource utilization rates of 60%, 50%, 70%, and 60%, respectively) to compete for GPU resources, resulting in a maximum GPU workload of up to 240%. This disordered competition slows down GPU processing speed and increases job completion time. A schematic diagram illustrating GPU usage using the oversubscription strategy of the interleaved packing scheme in this embodiment of the invention is shown below. Figure 4 As shown, over-optimization based on interleaved packing technology can effectively alleviate resource contention between tasks, with the GPU's maximum workload reduced to only 130%, and under ideal conditions, resource contention can be reduced by 50%. For example, without interleaved packing, when N tasks share the same GPU, at most N tasks may simultaneously contend for resources; however, with interleaved packing, multiple tasks form N / 2 pairs, so at any given time, at most only N / 2 tasks are in a resource contention state. The method provided in this embodiment of the invention distributes tasks in an orderly and alternating manner, reducing cross-task contention and improving GPU efficiency.

[0061] In practical implementation, Multi-Process Service (MPS) in related technologies can be used to enable GPUs to execute interleaved and packaged tasks in parallel on an over-allocated basis. This service facilitates GPU sharing among multiple processes. MPS allows multiple processes to share a single GPU context, eliminating the need for each process to have a separate context, thus significantly reducing context switching overhead. Furthermore, MPS uses a fair time-slicing method to allocate GPU computing resources to multiple tasks and determines the execution cycle of each task. This means that each task receives a specific allocation of computing time. This time-slicing scheduling allows processes packaged from multiple DLT tasks to pre-utilize GPU resources, thus laying the foundation for over-allocation.

[0062] Specifically, in some embodiments, the above method further includes:

[0063] The processor can simultaneously process one unassigned task that is in a phase across training cycles and another unassigned task that is in a phase within a training cycle. The processor includes a phase lock, which is used to prevent the processor from processing two or more unassigned tasks that are in the same task phase at the same time.

[0064] In specific implementation, the above methods are as follows: Figure 5 and Figure 6 As shown, in most cases, Task A and Task B will run in phases (intra-cycle phases and cross-cycle phases), and the phase that is currently running will be locked, while the phase that has been completed will be unlocked, such as... Figure 5 As shown; when a task (task B) completes a phase first, it will be blocked because other tasks (task A) have not yet released the lock on another phase, as... Figure 6 As shown.

[0065] In practical implementation, to determine how to interleave and package multiple tasks to be assigned, it is necessary to model the resource consumption and time of each DLT task. This embodiment of the invention uses a quintuple. To describe a DLT task, in the formula, Indicates task The number of training cycles, Indicates training task The number of parallel operations, i.e., the number of GPUs. Representing tasks Resource consumption in the three phases (initial phase, cross-cycle phase, and intra-cycle phase). In each phase, average CPU utilization C, maximum memory usage M, average GPU utilization G, maximum GPU memory usage GM, and required time T can be calculated. This formula represents...

[0066] This invention also establishes over-allocation constraints. For each GPU, an over-allocation rate is defined. The over-allocation rate sets the maximum workload factor for the device. GPU workload is quantified by its peak GPU utilization. Therefore, DLT tasks co-scheduled on the same GPU must satisfy the formula... The excess allocation constraint, in which This refers to a set of tasks running simultaneously on the same GPU. For GPU workloads, They are respectively The average GPU utilization across three phases (initial phase, cross-cycle phase, and intra-cycle phase). As a result, CPU and memory allocation is maintained at 100%, with over-allocation mitigating low utilization through user-specified resource redundancy and bloated GPU demand profiles.

[0067] In some embodiments, the process of generating an interleaved packaging scheme for at least two tasks to be assigned based on the task phase in S2 above includes:

[0068] For at least two tasks to be assigned, calculate the resource complementarity value between each pair of tasks, and add the packaging schemes of the two tasks with the highest resource complementarity values ​​to the interleaved packaging scheme; the formula for calculating the resource complementarity value includes:

[0069] ;

[0070] ;

[0071] in, Tasks to be assigned Tasks to be assigned The complementary value of resources The larger the value, the more obvious the resource complementarity between the two tasks. Used to indicate tasks to be assigned. Multiple calculation indicators, Used to indicate tasks to be assigned. Multiple calculation indicators, The function is used to calculate complementary values. Indicates taking and The maximum value in, , , , Tasks to be assigned Tasks to be assigned Several computational metrics include the number of training cycles. For two complementary tasks, it is best to... Tasks should be as close as possible to each other to avoid one task completing quickly in a shorter cycle while another runs alone, resulting in idle resources. Tasks to be assigned Tasks to be assigned The various computational metrics also include average CPU utilization C, maximum memory usage M, average GPU utilization G, maximum GPU memory usage GM, and time required across training cycle phases, as well as average CPU utilization C, maximum memory usage M, average GPU utilization G, maximum GPU memory usage GM, and time required within a single training cycle phase. Indicates tasks to be assigned In multiple computational metrics across training cycles, the above Indicates tasks to be assigned The above-mentioned computational metrics during the training period are... Indicates tasks to be assigned In multiple computational metrics across training cycles, the above Indicates tasks to be assigned During the training cycle, the sum of the resource requirements of the remaining five dimensions (C, M, G, GM, and T) for two complementary tasks in different staggered phases should be as close as possible to help reduce resource fluctuations.

[0072] In some embodiments, to avoid prolonging the JCT of DLT tasks with low GPU utilization fluctuations, this embodiment of the invention sets a matching factor for the tasks participating in the matching process. Prior to the step described above, which generates an interleaved packaging scheme for at least two tasks to be assigned based on the task phase, the following steps are also included:

[0073] The matching factor is calculated based on the duration of each task phase for each task to be assigned. In response to a matching factor greater than or equal to a preset matching factor threshold, an interleaved packaging scheme is generated for the task to be assigned; or, if the matching factor of the task to be assigned is less than the matching factor threshold, a separate processor allocation scheme is generated for the task to be assigned.

[0074] Specifically, since fluctuations in GPU resources are mainly caused by different tasks executed at different stages, this embodiment of the invention determines the aforementioned matching factor based on the duration of the task at different stages. Matching factor The calculation formulas include:

[0075] ;

[0076] in, As the matching factor, The duration of the training cycle phase for tasks to be assigned. The duration of a phase within a training cycle for tasks to be assigned.

[0077] In practice, the aforementioned matching factor threshold The value ranges from 0 to 0.5, with 0.16 being preferred.

[0078] This invention also provides pseudocode for the above-described complementary matching algorithm, as shown in Algorithm 1 below, wherein lines 1-5 of the pseudocode are used to base the algorithm on the above-described matching factor. Tasks that do not need to be packaged are removed, and the remaining tasks are packaged according to the number of GPUs. p Grouping is performed, and the code in lines 6 to 13 is based on the resource complementarity value. pair.

[0079] Algorithm 1: Complementary Matching Algorithm

[0080] Input: Matching factor threshold The set of tasks to be assigned, S;

[0081] 1: Initialize an empty set of unmatched tasks D and an empty list M;

[0082] 2: Execute the following on task i in set S:

[0083] 3: Calculate the matching factor for task i ;

[0084] 4: If ,but:

[0085] 5: Add task i to set D, where the key of task i is the GPU value;

[0086] 6: Execute the following on the key in set D:

[0087] 7: The set of unmatched keys S = D[key];

[0088] 8: Calculate the resource complementarity value for all tasks i and j in set S. ;

[0089] 9: Put all ( Add task i and task j to list L;

[0090] 10: According to Sort the contents of list L in descending order;

[0091] 11: For the (in list L) Task i and Task j) are executed:

[0092] 12: If task i and task j are not in list M, then:

[0093] 13: Add (task i, task j) to list M;

[0094] 14: Return to list M.

[0095] For the packaged tasks, when allocating processors, the aforementioned multi-GPU orchestration algorithm needs to consider both which tasks to prioritize for processor allocation and which processors to prioritize for each task. The following embodiments of the present invention will provide specific implementation methods for processor allocation. It should be noted that in the following embodiments, two interleaved packaged tasks are treated as one task and their corresponding processors are allocated accordingly.

[0096] In related technologies, some schedulers employ strategies such as First-In-First-Out (FIFO), Shortest Remaining Time First (SRTF), and Shortest Remaining Service First (SRSF). However, these strategies often encounter common queuing and head-of-service (HOL) problems during scheduling. Here, HOL specifically refers to resource-related blocking, meaning that tasks with low resource requirements may be blocked by tasks with high resource requirements. However, simply skipping larger tasks can lead to task starvation, thereby increasing the task completion time (JCT). This problem persists even under simple over-allocation.

[0097] The embodiments of the present invention employ a bucket-based (equivalent to a queue) scheduling method. Non-blocking scheduling is used within the buckets to effectively alleviate HOL blocking. Meanwhile, blocking scheduling is still used between queues to prevent starvation.

[0098] In some embodiments, the steps of allocating processors to multiple tasks using blocking scheduling and non-blocking scheduling include:

[0099] Based on the arrival time of the tasks to be assigned, at least two tasks should be placed into different time buckets. In practice, this can be done using the formula... Where, Bid is the ID of the bucket to which the task is assigned, and the floor function is used to round down. For the task arrival time, This is the length of the bucket.

[0100] Different time buckets are sorted from earliest to latest according to their arrival time, and within each time bucket, tasks to be assigned are sorted from highest to lowest priority. Specifically, this priority can be obtained according to the SRSF method described above, or it can be obtained through other methods; this embodiment of the invention does not impose excessive limitations.

[0101] Processors are allocated sequentially for each sorted task to be assigned, wherein, in response to the time bucket, processors are allocated in a non-blocking scheduling manner, i.e., if a task to be assigned at the beginning of the time bucket cannot be allocated a processor, the task to be assigned is skipped; and, in response to the time bucket, processors are allocated in a blocking scheduling manner, i.e., if all tasks to be assigned in the time bucket at the beginning of the time bucket cannot be allocated a processor, the allocation is paused until all tasks to be assigned in the time bucket are allocated processors.

[0102] Figure 7 This document illustrates an application scenario of the method described above in a specific embodiment. In this scenario, multi-GPU parallel tasks are designed to utilize only 40% of the capacity of each GPU. Tasks are grouped into two buckets based on arrival time and prioritized using the SRSF method. Only four GPUs are available in the GPU cluster. Therefore, the highest priority task (5,1) in bucket 1 requires five GPUs and cannot be executed. The second highest priority task (2,5) in bucket 1 only requires two GPUs, so it is prioritized for execution on the GPU of node b. At this point, two GPUs remain available in the GPU cluster, while all other tasks in bucket 1 require more than two GPUs and are therefore still blocked. Although there are enough GPUs available for the highest priority task (1,1) in bucket 2, this task is also blocked because there are still unassigned tasks in bucket 1. This method prevents a complete HOL block caused by task (5,1) while ensuring that task (5,1) is not starved.

[0103] In related technologies, for multi-GPU parallel tasks, the fragmentation problem of being allocated on different GPUs can also lead to latency issues. This is because multi-GPU parallel tasks need to be synchronized between GPUs, and direct over-allocation cannot ensure that the task receives the same resources on all GPUs. This difference may lead to changes in execution speed between different GPUs, causing faster GPUs to wait for slower GPUs, thus significantly increasing JCT. Figure 8 The diagram illustrates the result of a typical greedy strategy for CPU selection in related techniques, which prioritizes allocating multi-GPU parallel tasks to idle GPUs. However, this approach leads to uneven deployment, failing to accelerate parallel tasks and causing GPU resource fragmentation, which adversely affects the execution of other tasks.

[0104] In view of this, in some embodiments, multiple processors are distributed across multiple nodes, each node including at least one processor; before allocating processors to multiple tasks to be assigned using blocking and non-blocking scheduling methods, the selection of GPUs is also included, specifically including:

[0105] Select a subset of processors as available processors based on the remaining resources of each processor.

[0106] Multiple nodes are sorted first according to the number of available processors from highest to lowest. Then, nodes with the same number of available processors are sorted a second time according to the total remaining resources of available processors within the node.

[0107] Within each node, multiple available processors are sorted from highest to lowest according to their remaining resource quantity, resulting in a sorted result of all available processors across all nodes.

[0108] Processors are allocated to multiple tasks using both blocking and non-blocking scheduling methods, including:

[0109] Available processors are allocated to multiple tasks to be assigned according to their sorting results using both blocking and non-blocking scheduling methods.

[0110] The method described in this embodiment of the invention, by rationally prioritizing available processors based on remaining resources, enables multi-GPU parallel tasks to be allocated to GPUs with similar remaining resource space during processor selection, thereby alleviating task fragmentation and reducing cross-server computation. The result of CPU selection based on the above method of the present invention is as follows: Figure 9 As shown, this method fully utilizes GPU resources, prevents resource fragmentation, and alleviates the fragmentation problem. Meanwhile, in Figure 7 In the embodiment shown, the fragmentation problem for task (2,5) is also mitigated due to GPU load balancing in node b.

[0111] This invention also provides pseudocode for the above-mentioned multi-GPU orchestration algorithm, as shown in Algorithm 2 below.

[0112] Algorithm 2: Multi-GPU orchestration algorithm.

[0113] Input: Bucket length µ, DLT task bucket Q;

[0114] 1: Initialize an empty GPU sort list W;

[0115] 2: Perform the following operations on task i in bucket Q:

[0116] 3: Get the Bid value of task i (Bid is the ID value of the bucket, used to record the buckets that have not been scheduled).

[0117] 4: Add task i to the bucket S corresponding to the Bid value;

[0118] 5: Sort all buckets S in ascending order of their Bid values;

[0119] 6: Define the bucket flag f=False and the bucket ID b=None (f is a flag bit used in conjunction with the bucket ID to record the first bucket that has not been fully scheduled).

[0120] 7: For task set S, perform the following operations:

[0121] 8: Use Algorithm 1 to obtain the matching list M of set S;

[0122] 9: Sort the tasks in list M using the SRSF algorithm;

[0123] 10: For interleaved and packaged tasks Jm=(Jobi, Jobj), perform the following operations in list M:

[0124] 11: On node x, obtain the number of Nx GPUs that satisfy the resource constraints of Jm;

[0125] 12: Sort nodes in descending order using Nx and the percentage of remaining resources;

[0126] 13: If If x ∈ all nodes, then:

[0127] 14: Select the first p GPUs based on node sorting;

[0128] 15: Add the selected GPU to the sort list W;

[0129] 16: Modify the remaining resources of the selected GPU;

[0130] 17: Otherwise;

[0131] 18: If f == False, then:

[0132] 19: f=True, b=Bid (used to record the ID of the bucket where the first scheduling failure occurred);

[0133] 20: Otherwise, if b! = Bid, then:

[0134] 21: Return the sorted list W (If the current bucket is not the same as the bucket at the time of the first scheduling failure when the second scheduling fails, stop scheduling to prevent tasks that failed to be scheduled in the first bucket from being left unscheduled).

[0135] 22: Return the sorted list W.

[0136] The architecture of the HyperWeave scheduler provided in this embodiment of the invention is as follows: Figure 10 As shown, the system is built on a master-slave node model. The master node is responsible for analyzing tasks, monitoring cluster resources, and scheduling tasks, i.e., executing the methods S1 to S3 described above. The slave nodes are responsible for executing tasks and uploading the task completion status to the master node. Specifically, the master node includes three components: a task analyzer, a cluster monitor, and a task scheduler, while the slave nodes include a task executor component.

[0137] The task analyzer categorizes tasks into initial phases, inter-cycle phases, and intra-cycle phases by evaluating the time and resource consumption at different stages of the assigned task, i.e., by executing the method described in S1 above. The task analyzer executes this method by running several batches, thereby recording and inferring the resource and time consumption at different stages.

[0138] The cluster monitor tracks resource information and task status within the cluster. Resource information includes the resource capacity and remaining resources for each processor. Resource capacity is used to assess the feasibility of an interleaved packing scheme, while remaining resources are used to identify available worker nodes and processors for task execution. Upon task completion, the cluster monitor logs the event and updates available resources for future scheduling.

[0139] The task scheduler uses a complementary matching algorithm to generate an interleaved packing scheme and uses a multi-GPU orchestration algorithm to dynamically map tasks to appropriate processors, i.e., executes the methods S2 and S3 described above.

[0140] The task executor executes a specified DLT task on a designated GPU, packaging and executing the task in parallel using an interleaved packaging scheme. The task executor requires the GPU to call the aforementioned Multi-Process Service (MPS) to implement GPU context operation.

[0141] The embodiments of this invention experimentally verified the data differences between the HyperWeave scheduler and other schedulers in related technologies (including FIFO, SRSF, MPS, SRTF, Muri, and Lucid) in terms of average task completion time, maximum task completion time, and task waiting time. Figures 11 to 13 As shown, where, Figure 11 The vertical axis represents the average task completion time. Figure 12 The vertical axis represents the maximum task completion time. Figure 13 The vertical axis represents the task waiting time. Figures 11 to 13 The x-axis represents different task traces. The results show that the method and scheduler provided in this embodiment of the invention reduce the average task completion time by 2.93 to 3.97 times, the maximum task completion time by 1.97 to 2.21 times, and the task waiting time by 5.76 to 8.66 times compared with other schedulers in related technologies.

[0142] Figure 14 The figure illustrates the GPU resource usage of the HyperWeave scheduler provided in this embodiment of the invention and other schedulers in related technologies (including FIFO, SRSF, MPS, SRTF, Muri, and Lucid) during task execution. The horizontal axis represents time, and the vertical axis represents GPU resource utilization. It can be clearly seen from the figure that HyperWeave is significantly better than other schedulers in terms of resource utilization.

[0143] Figure 15 The figure illustrates the GPU allocation of the HyperWeave scheduler provided in this embodiment of the invention and other schedulers (including FIFO, SRSF, MPS, SRTF, Muri, and Lucid) in related technologies before task execution. The horizontal axis represents time, and the vertical axis represents the GPU allocation rate. As shown in the figure, when ξ is set to 200%, HyperWeave achieves a maximum GPU allocation rate of nearly 200%, while the maximum GPU allocation rate of other schedulers is still 100%.

[0144] Figure 16 The figure illustrates the task waiting bucket length during task allocation for the HyperWeave scheduler provided in this embodiment of the invention and other schedulers in related technologies (including FIFO, SRSF, MPS, SRTF, Muri, and Lucid). The horizontal axis represents time, and the vertical axis represents the task waiting bucket length. As shown in the figure, in the HyperWeave scheduler, the bucket length decreases rapidly, indicating that the system has a strong job processing capability, which helps to reduce task waiting time.

[0145] Figure 17 The figure illustrates the GPU allocation of the HyperWeave scheduler provided in this embodiment of the invention and other schedulers in related technologies (including FIFO, SRSF, MPS, SRTF, Muri, Lucid, and Tiresias) during actual task execution. The horizontal axis represents time, and the vertical axis represents the actual GPU allocation rate. As shown in the figure, HyperWeave achieves a maximum GPU allocation rate of nearly 200%, while the maximum GPU allocation rate of other schedulers is still 100%.

[0146] This invention also includes an experiment on the optimal parameter settings for the HyperWeave scheduler.

[0147] The embodiments of the present invention have achieved optimal over-allocation rates for different memory types and different GPU models. The experiment involved three GPUs: a P100 with 16GB of VRAM, a V100 with 32GB of VRAM, and an A100 with 80GB of VRAM. The results showed that the optimal over-allocation rate was [not specified in the original text]. The value is approximately 200%.

[0148] The embodiments of the present invention are also based on different matching factor thresholds. Simulation experiments were conducted. The value ranges from 0 to 0.5, adjusted in steps of 0.02. The experimental results show that when the matching factor threshold... When the JCT increases, it first decreases, then increases, and eventually stabilizes. In HyperWeave, although interleaving improves resource utilization, improper pairing of two jobs can also lead to an increase in JCT. Therefore, adjusting the matching factor threshold is crucial. However, it's crucial not to simply increase the value indiscriminately; an appropriate value must be chosen. After numerous experiments, a matching factor threshold was selected. As the optimal value.

[0149] The embodiments of the present invention are also based on different barrel lengths. A simulation experiment was conducted to determine the length of the bucket. The range is from 1×10 3 Up to 1×10 7 The experimental results show that as the bucket length increases, JCT and other performance metrics decrease. This phenomenon is easy to understand: a longer bucket length provides a wider range of non-blocking scheduling options, thereby reducing the GPU's idle percentage and improving overall efficiency. In practical applications, the bucket length needs to be determined based on specific requirements. When the bucket length is infinitely long, the job scheduling order in HyperWeave will be the same as SRSF. This embodiment of the invention uses a 1×10... 6 As a balance, this ensures that HyperWeave can achieve good system performance while preventing task starvation.

[0150] This invention also conducted ablation experiments on key technical solutions (complementary matching algorithm and multi-GPU orchestration algorithm). First, the results of experiments using both the complementary matching algorithm and the multi-GPU orchestration algorithm were standardized (i.e., all results were 1). Then, after replacing the multi-GPU orchestration algorithm with a basic orchestration strategy (FIFO) (while retaining the complementary matching algorithm), the average task completion time was 1.1, the maximum task completion time was 1.16, and the task waiting time was 1.24. Using only the multi-GPU orchestration algorithm, the average task completion time was 1.44, the maximum task completion time was 0.96, and the task waiting time was 1.74. This demonstrates that it is the synergistic effect of the interleaved execution and orchestration techniques provided by this invention that enables HyperWeave to achieve superior performance.

[0151] It should be noted that the method of this embodiment can be executed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method of this embodiment, and the multiple devices will interact with each other to complete the above method.

[0152] It should be noted that the above description describes some embodiments of the present invention. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in a different order than that shown in the above embodiments and still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0153] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this invention also provides a graphics processor overload allocation device, such as... Figure 18 As shown, it includes:

[0154] The task analysis module 10 is used to obtain multiple tasks to be assigned and the resource consumption data of each task to be assigned.

[0155] The task scheduling module 20 is used to divide multiple tasks to be assigned into task stages based on the resource consumption data of each task to be assigned, and to generate an interleaved packaging scheme for at least two tasks to be assigned based on the task stages. The interleaved packaging scheme includes enabling one processor to process at least two tasks to be assigned simultaneously. The task scheduling module 20 is also used to allocate processors to multiple tasks to be assigned based on the interleaved packaging scheme and the arrival time of the tasks to be assigned, using both blocking and non-blocking scheduling methods. At least one processor is allocated tasks exceeding the total resource capacity of that processor.

[0156] This invention provides a GPU overload task allocation scheduler, hereinafter referred to as HyperWeave, for executing the aforementioned GPU overload task allocation method. First, this invention provides a novel complementary matching algorithm (executed by the task analysis module 10 and the task scheduling module 20) for job packaging to optimize GPU sharing in the time dimension. This algorithm identifies tasks with complementary resource requirements and packages and binds them to alleviate resource contention in the time dimension. Second, this invention introduces a novel multi-GPU orchestration algorithm (executed by the task scheduling module 20). This algorithm dynamically allocates GPUs to tasks based on their arrival times, combining blocking and non-blocking scheduling methods to avoid head-blocking and lagging issues in the spatial dimension. Simultaneously, this invention allows for overload task allocation to processors, meaning that at least one processor is allocated more tasks than the total resource capacity of that processor. Combined with the aforementioned task alternation execution mechanism, GPU utilization is improved by allowing tasks to temporarily overlap their resource requirements. In related technologies, over-allocation is essentially a free competition for resources among tasks. Severe resource competition will lead to an increase in job completion time. The embodiments of the present invention avoid the head blocking problem through the above-mentioned multi-GPU orchestration algorithm, and at the same time, can also prevent tasks from being starved.

[0157] An embodiment of the invention provides a graphics processor overload allocation device for reference. Figure 10 The task analysis module 10 is the task analyzer mentioned above, and the task scheduling module 20 is the task scheduler mentioned above.

[0158] For ease of description, the above apparatus is described in terms of its functions, divided into various modules. Of course, in implementing this invention, the functions of each module can be implemented in one or more software and / or hardware components.

[0159] The apparatus of the above embodiments is used to implement the corresponding graphics processor excess task allocation method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0160] Figure 19A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention is shown.

[0161] like Figure 19 As shown, the electronic device may include a processor 1101 and a memory 1102 storing computer program instructions.

[0162] Specifically, the processor 1101 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of the present invention.

[0163] Memory 1102 may include a mass storage device for information or instructions. For example, and not limitingly, memory 1102 may include a hard disk drive (HDD), a floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 1102 may include removable or non-removable (or fixed) media. Where appropriate, memory 1102 may be internal or external to the integrated gateway device. In a particular embodiment, memory 1102 is a non-volatile solid-state memory. In a particular embodiment, memory 1102 includes read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable programmable PROM (EEPROM), an electrically alterable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0164] The processor 1101 reads and executes computer program instructions stored in the memory 1102 to perform the steps of the graphics processor overload allocation method provided in this embodiment of the invention.

[0165] In one example, the electronic device may also include a transceiver 1103 and a bus 1104. Wherein, as... Figure 19 As shown, the processor 1101, memory 1102 and transceiver 1103 are connected via bus 1104 and communicate with each other.

[0166] Bus 1104 may include hardware, software, or both. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industrial Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, bus 1104 may include one or more buses. Although specific buses are described and illustrated in the embodiments of the present invention, the present invention is contemplated by any suitable bus or interconnect.

[0167] The following are embodiments of a computer-readable storage medium provided in this invention. This computer-readable storage medium and the graphics processor overload allocation method of the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the computer-readable storage medium, please refer to the embodiments of the graphics processor overload allocation method described above.

[0168] This embodiment provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform a graphics processor overload allocation method.

[0169] Of course, the computer-executable instructions provided in the embodiments of the present invention are not limited to the above-described method operations, but can also perform related operations in the graphics processor overload allocation method provided in any embodiment of the present invention.

[0170] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer cloud platform (which can be a personal computer, server, or network cloud platform, etc.) to execute the graphics processor overload allocation method provided in the various embodiments of the present invention.

[0171] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the aforementioned element.

[0172] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments described above, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for allocating excess tasks to a graphics processing unit, characterized in that, include: Obtain multiple tasks to be assigned and resource consumption data for each task to be assigned; The multiple tasks to be assigned are divided into task stages based on the resource consumption data of each task to be assigned, and an interleaved packaging scheme is generated for at least two tasks to be assigned based on the task stages. The interleaved packaging scheme includes enabling one processor to process at least two tasks to be assigned simultaneously. Based on the interleaved packaging scheme, and according to the arrival time of the tasks to be assigned, processors are allocated to multiple tasks to be assigned through blocking scheduling and non-blocking scheduling, wherein at least one processor is allocated more tasks to be assigned than the total resource capacity of that processor. The resource consumption data includes the resource consumption of the tasks to be assigned at different times; The task to be assigned is divided into stages based on its resource consumption data, including: Based on the resource consumption of the task to be assigned at different times, the task phase of the task to be assigned is divided into a phase across training cycles and a phase within training cycles. For at least one resource, the resource consumption of the phase within training cycles is greater than the resource consumption of the phase across training cycles. The staggered packaging scheme also includes: One of the processors can simultaneously process one task to be assigned that is in the cross-training cycle phase and another task to be assigned that is in the phase within the training cycle. Based on the arrival time of the tasks to be assigned, processors are allocated to multiple tasks to be assigned using the blocking scheduling method and the non-blocking scheduling method, including: Based on the arrival time of the tasks to be assigned, at least two tasks to be assigned are placed into different time buckets. The different time buckets are sorted from earliest to latest according to their arrival time, and within each time bucket, the tasks to be assigned are sorted from highest to lowest priority. A processor is sequentially assigned to each sorted task to be assigned, wherein, in response to a task that cannot be assigned a processor within the time bucket, the task to be assigned is skipped; and, in response to all tasks in the time bucket that cannot be assigned a processor between the time buckets, the assignment is paused until all tasks in the time bucket have been assigned a processor.

2. The graphics processor excess task allocation method as described in claim 1, characterized in that, The generation of an interleaved packaging scheme for at least two tasks to be assigned based on the task stage includes: For at least two tasks to be assigned, calculate the resource complementarity value between each pair of tasks to be assigned, and add the packaging scheme of the two tasks to be assigned with the highest resource complementarity value to the interleaved packaging scheme; The formula for calculating the resource complementarity value includes: ； ； in, Tasks to be assigned Tasks to be assigned The complementary value of resources Used to indicate tasks to be assigned. Multiple calculation indicators, Used to indicate tasks to be assigned. Multiple calculation indicators, The function is used to calculate complementary values. Indicates taking and The maximum value in the list, the task to be assigned Tasks to be assigned The various computational metrics include the number of training cycles, the average CPU utilization across the training cycle phases, the maximum memory usage, the average GPU utilization, the maximum GPU memory usage, the required time, and the average CPU utilization, maximum memory usage, average GPU utilization, maximum GPU memory usage, and duration of each phase within the training cycle.

3. The method for allocating excess tasks to a graphics processor as described in claim 1, characterized in that, Before generating an interleaved packaging scheme for at least two tasks to be assigned based on the task phase, the method further includes: The matching factor is calculated based on the duration of each task phase of each pending task; In response to the matching factor of the task to be assigned being greater than or equal to a preset matching factor threshold, the interleaved packaging scheme is generated for the task to be assigned; or, if the matching factor of the task to be assigned is less than the matching factor threshold, the processor allocation scheme is generated separately for the task to be assigned. The formula for calculating the matching factor includes: ； in, Tasks to be assigned The matching factor, Tasks to be assigned The duration of the aforementioned cross-training cycle phase, Tasks to be assigned The duration of a phase within the training cycle.

4. The method for allocating excess tasks to a graphics processor as described in claim 1, characterized in that, The processors are distributed across multiple nodes, and each node includes at least one of the processors. Before allocating processors to the multiple tasks to be assigned using the blocking scheduling method and the non-blocking scheduling method, the process further includes: Select a subset of processors as available processors based on the remaining resources of each processor; Multiple nodes are sorted from highest to lowest according to the number of available processors, and nodes with the same number of available processors are sorted a second time from highest to lowest according to the total remaining resources of available processors within the node. Within each node, the available processors are sorted from highest to lowest according to their remaining resource quantity, resulting in a sorting result for all available processors across all nodes. The process of allocating processors to multiple tasks through blocking and non-blocking scheduling includes: The available processors are allocated to multiple tasks to be assigned according to the sorting result using both blocking and non-blocking scheduling methods.

5. The graphics processor excess task allocation method as described in claim 1, characterized in that, it further... include: The processor can simultaneously process one task to be assigned that is in the cross-training cycle phase and another task to be assigned that is in the intra-training cycle phase, wherein the processor includes a phase lock, the phase lock being used to prevent the processor from simultaneously processing two or more tasks to be assigned that are in the same task phase.

6. A graphics processor overload allocation device, characterized in that, include: The task analysis module is used to obtain multiple tasks to be assigned and the resource consumption data of each task to be assigned; The task scheduling module is used to divide multiple tasks to be assigned into task stages according to the resource consumption data of each task to be assigned, and to generate an interleaved packaging scheme for at least two tasks to be assigned based on the task stages, wherein the interleaved packaging scheme includes enabling one processor to process at least two tasks to be assigned simultaneously. The task scheduling module is also used to allocate processors to multiple tasks to be assigned based on the interleaved packaging scheme and according to the arrival time of the tasks to be assigned, through blocking scheduling and non-blocking scheduling, wherein at least one of the processors is allocated more tasks to be assigned than the total resource capacity of that processor. The resource consumption data includes the resource consumption of the tasks to be assigned at different times; The task scheduling module is also used for: Based on the resource consumption of the task to be assigned at different times, the task phase of the task to be assigned is divided into a phase across training cycles and a phase within training cycles. For at least one resource, the resource consumption of the phase within training cycles is greater than the resource consumption of the phase across training cycles. The staggered packaging scheme also includes: One of the processors can simultaneously process one task to be assigned that is in the cross-training cycle phase and another task to be assigned that is in the phase within the training cycle. The task scheduling module is also used for: Based on the arrival time of the tasks to be assigned, at least two tasks to be assigned are grouped into different time buckets; the different time buckets are sorted from earliest to latest according to their arrival time, and within each time bucket, the tasks to be assigned are sorted from highest to lowest priority; a processor is assigned to each sorted task to be assigned in sequence, wherein, in response to a task to be assigned that is ranked higher within a time bucket and cannot be assigned a processor, that task to be assigned is skipped; and, in response to a situation between time buckets where all tasks to be assigned in a time bucket ranked higher cannot be assigned a processor, the assignment is paused until all tasks to be assigned in that time bucket have been assigned a processor.

7. An electronic device, characterized in that, include: processor; A memory for storing executable instructions; wherein the processor is configured to read the executable instructions from the memory and execute the executable instructions to implement the method of any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, The storage medium stores a computer program that, when executed by a processor, causes the processor to implement the method described in any one of claims 1 to 5.

Images