Graphics processor excess task allocation method and device

Abstract

The invention provides an excess task allocation method and device for a graphics processor, and the method comprises the steps: dividing task stages for a plurality of to-be-allocated tasks according to the resource consumption data of each to-be-allocated task, and generating a staggered packaging scheme for at least two to-be-allocated tasks based on the task stages, so as to optimize the GPU sharing in the time dimension, the algorithm identifies tasks with complementary resource requirements, and packs and binds the tasks to relieve resource competition in time dimension; based on a staggered packaging scheme, according to the arrival time of the to-be-allocated task, the GPU is allocated to the to-be-allocated task by adopting a mode of combining a blocking scheduling mode and a non-blocking scheduling mode, so that the problems of head blocking and lagging behind in a spatial dimension are avoided; wherein at least one processor is allocated with to-be-allocated tasks exceeding the total resource capacity of the processor, and the GPU utilization rate is improved by allowing the tasks to temporarily overlap the resource requirements of the tasks in combination with a task alternate execution mechanism.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computer, and particularly relates to a graphics processor excess task allocation method and device. BACKGROUND

[0002] With the continuous growth of Deep Learning (DL) models in complexity and scale, the demand for computing resources, particularly GPUs (Graphics Processing Units), is increasing. As a result, training in a cloud environment is becoming more common. Users typically submit Deep Learning Training (DLT) jobs to the cloud, and the cloud service allocates resources and executes based on the availability of GPUs.

[0003] During the execution of DLT, redundant resource allocation and dynamic resource usage result in low GPU utilization, which in turn leads to longer Job Completion Time (JCT). For DLT job scheduling, related technologies usually require that the total resource requirement of DLT jobs on a GPU cannot exceed the capacity of the GPU. However, due to the difficulty of perfectly matching multiple DLT jobs to the full load of a GPU, the above restrictions often exacerbate resource waste. SUMMARY

[0004] Based on the above technical problems, the application provides a graphics processor excess task allocation method and device.

[0005] The application provides a graphics processor excess task allocation method, comprising: obtaining a plurality of to-be-allocated tasks and resource consumption data of each to-be-allocated task; dividing task stages for the plurality of to-be-allocated tasks according to the resource consumption data of each to-be-allocated task, and generating an interleaved packaging scheme for at least two to-be-allocated tasks based on the task stages, wherein the interleaved packaging scheme comprises causing one processor to simultaneously process at least two to-be-allocated tasks; based on the interleaved packaging scheme and according to the arrival time of the to-be-allocated tasks, allocating processors for the plurality of to-be-allocated tasks by using a blocking scheduling mode and a non-blocking scheduling mode, wherein at least one processor is allocated with to-be-allocated tasks that exceed the total resource capacity of the processor.

[0006] Optionally, the resource consumption data of the to-be-assigned tasks comprises resource consumption amounts of the to-be-assigned tasks at different times; the task stages of the to-be-assigned tasks are divided according to the resource consumption data of the to-be-assigned tasks, comprising: the task stages of the to-be-assigned tasks are divided into a cross-training cycle stage and a training cycle stage according to the resource consumption amounts of the to-be-assigned tasks at different times, wherein, for at least one resource, the resource consumption amount of the training cycle stage is greater than the resource consumption amount of the cross-training cycle stage; the interleaved packaging scheme comprises: one of the processors simultaneously processes one to-be-assigned task in the cross-training cycle stage and another to-be-assigned task in the training cycle stage.

[0007] Optionally, the interleaved packaging scheme is generated for the at least two to-be-assigned tasks based on the task stages, comprising: for the at least two to-be-assigned tasks, a resource complementarity value between the to-be-assigned tasks is calculated, and the packaging scheme of the two to-be-assigned tasks with the highest resource complementarity value is added to the interleaved packaging scheme; the calculation formula of the resource complementarity value comprises: ; wherein, is the resource complementarity value of the to-be-assigned task and the to-be-assigned task is used to represent the plurality of computing indicators of the to-be-assigned task is used to represent the plurality of computing indicators of the to-be-assigned task is a complementarity value calculation function, represents the maximum value in and , the plurality of computing indicators of the to-be-assigned task and the to-be-assigned task each comprise a training cycle number, an average CPU usage of the cross-training cycle stage, a maximum memory usage, an average GPU usage, a maximum GPU memory usage, a required time, and an average CPU usage, a maximum memory usage, an average GPU usage, a maximum GPU memory usage, and a duration of the training cycle stage.

[0008] Optionally, before the interleaved packaging scheme is generated for the at least two to-be-assigned tasks based on the task stages, it further comprises: a matching factor is calculated according to the duration of each task stage of each to-be-assigned task; in response to the matching factor of a to-be-assigned task being greater than or equal to a preset matching factor threshold, the interleaved packaging scheme is generated for the to-be-assigned task; or, in response to the matching factor of a to-be-assigned task being less than the matching factor threshold, a processor allocation scheme is generated separately for the to-be-assigned task; the calculation formula of the matching factor comprises: ; wherein,​​​​ for the to-be-assigned task the above matching factor, for the to-be-assigned task the above duration of the cross-training cycle phase, for the to-be-assigned task the above duration of the training cycle phase.

[0009] Optionally, the plurality of to-be-assigned tasks are assigned to the plurality of processors by the blocking scheduling mode and the non-blocking scheduling mode according to the arrival time of the to-be-assigned tasks, comprising: grouping at least two to-be-assigned tasks into different time buckets according to the arrival time of the to-be-assigned tasks; sorting the different time buckets according to the arrival time from early to late, and sorting the to-be-assigned tasks in each time bucket according to the priority from high to low; and assigning the processors to each to-be-assigned task in turn after sorting, wherein, in response to the to-be-assigned task sorted in the front being unable to be assigned to the processor in the time bucket, the to-be-assigned task is skipped; and in response to all to-be-assigned tasks in the time bucket sorted in the front being unable to be assigned to the processor, the assignment is suspended until all to-be-assigned tasks in the time bucket are assigned to the processor.

[0010] Optionally, the plurality of processors are distributed in a plurality of nodes, and each node includes at least one processor; before the plurality of to-be-assigned tasks are assigned to the plurality of processors by the blocking scheduling mode and the non-blocking scheduling mode, the method further comprises: selecting part of the processors as available processors according to the resource remaining amount of each processor; sorting the plurality of nodes according to the number of available processors from high to low once, and sorting the nodes with the same number of available processors according to the total resource remaining amount of the available processors in the node from high to low twice; sorting the plurality of available processors in each node according to the resource remaining amount from high to low to obtain the sorting result of all available processors in all nodes; and the plurality of to-be-assigned tasks are assigned to the plurality of processors by the blocking scheduling mode and the non-blocking scheduling mode according to the sorting result.

[0011] Optionally, the method further comprises: simultaneously processing one to-be-assigned task in the cross-training cycle phase and another to-be-assigned task in the training cycle phase by the processor, wherein the processor includes a phase lock, and the phase lock is used to prohibit the processor from simultaneously processing two or more to-be-assigned tasks in the same task phase.

[0012] Based on the same inventive concept, the application further provides a graphic processor excess task allocation device, comprising: a task analysis module, configured to obtain a plurality of to-be-allocated tasks and resource consumption data of each to-be-allocated task; a task scheduling module, configured to divide a task stage for the plurality of to-be-allocated tasks according to the resource consumption data of each to-be-allocated task, and generate an interleaving packaging scheme for at least two to-be-allocated tasks based on the task stage, wherein the interleaving packaging scheme comprises causing one processor to simultaneously process at least two to-be-allocated tasks; and the task scheduling module is further configured to allocate the processors for the plurality of to-be-allocated tasks based on the interleaving packaging scheme and according to the arrival time of the to-be-allocated tasks by using a blocking scheduling mode and a non-blocking scheduling mode, wherein at least one processor is allocated with to-be-allocated tasks whose resource capacity sum exceeds the resource capacity of the processor.

[0013] Based on the same inventive concept, the application further provides an electronic device, comprising: a processor; a memory configured to store executable instructions; wherein the processor is configured to read the executable instructions from the memory and execute the executable instructions to implement any one of the methods.

[0014] Based on the same inventive concept, the application further provides a computer readable storage medium having a computer program stored thereon, wherein the storage medium stores the computer program, and when the computer program is executed by a processor, the processor implements any one of the methods. The application has the following beneficial effects: the application provides a new complementary matching algorithm for packaging jobs to optimize GPU sharing in the time dimension, the algorithm identifies tasks with complementary resource requirements and binds them to relieve resource competition in the time dimension. Secondly, the application introduces a new multi-GPU orchestration algorithm, which dynamically allocates GPUs for to-be-allocated tasks according to the arrival time of the to-be-allocated tasks, in combination with the blocking scheduling mode and the non-blocking scheduling mode, to avoid head blocking and straggler problems in the space dimension. At the same time, the application allows to allocate tasks in excess of the capacity of the processor, in combination with the above task alternation execution mechanism, to improve GPU utilization by allowing temporary overlap of resource requirements of tasks. BRIEF DESCRIPTION OF DRAWINGS

[0015] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the application and, together with the description, serve to explain the principles of the application.

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the accompanying drawings needed in the embodiments or prior art description will be briefly introduced as follows. Obviously, for those skilled in the art, other drawings can also be obtained based on these drawings without creative labor.

[0017] Figure 1 A flowchart of a graphics processor over-subscription method according to an embodiment of the present application is shown in FIG. 3; Figure 2 A resource requirement diagram of a task to be allocated in different stages according to an embodiment of the present application is shown in FIG. 4; Figure 3 A basic over-submission strategy in the related art is shown in FIG. 5; Figure 4 An over-subscription strategy based on an interleaving packing scheme according to an embodiment of the present application is shown in FIG. 6; Figure 5 A diagram of a processor processing an interleaving packed task according to an embodiment of the present application is shown in FIG. 7; Figure 6 Another diagram of a processor processing an interleaving packed task according to an embodiment of the present application is shown in FIG. 8; Figure 7 A multi-GPU orchestration method according to an embodiment of the present application is shown in FIG. 9; Figure 8 A result diagram of CPU selection based on a typical greedy strategy in the related art is shown in FIG. 10; Figure 9 A result diagram of CPU selection based on a multi-GPU orchestration method according to an embodiment of the present application is shown in FIG. 11; Figure 10 An architecture diagram of a HyperWeave scheduler according to an embodiment of the present application is shown in FIG. 12; Figure 11 A result diagram of a simulation comparison experiment between an embodiment of the present application and the related art is shown in FIG. 13; Figure 12 A result diagram of a simulation comparison experiment between another embodiment of the present application and the related art is shown in FIG. 14; Figure 13 A result diagram of a simulation comparison experiment between another embodiment of the present application and the related art is shown in FIG. 15; Figure 14 A result diagram of a simulation comparison experiment between another embodiment of the present application and the related art is shown in FIG. 16; Figure 15 A result diagram of a simulation comparison experiment between another embodiment of the present application and the related art is shown in FIG. 17; Figure 16 A result diagram of a simulation comparison experiment between another embodiment of the present application and the related art is shown in FIG. 18; Figure 17 A result diagram of a simulation comparison experiment between another embodiment of the present application and the related art is shown in FIG. 19; Figure 18 A structure diagram of a graphics processor over-subscription device according to an embodiment of the present application is shown in FIG. 20. Figure 19 A schematic diagram of an electronic device hardware structure is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0018] In order to enable a more clear understanding of the above-mentioned purposes, features and advantages of the embodiments of the present application, the schemes of the embodiments of the present application will be further described below. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0019] In the following description, a large number of specific details are set forth in order to facilitate a thorough understanding of the embodiments of the present application, but the embodiments of the present application can also be implemented in other manners different from those described herein; obviously, the embodiments described in the specification are only a part of the embodiments of the present application, and not all the embodiments.

[0020] The present application tracks the use of GPUs in related technologies and finds that 81.75% of DLT tasks run on a single GPU, and efficient GPU sharing is crucial to cluster performance. As the performance of GPUs improves, a single GPU can accommodate more DLT jobs, but related technologies use a conservative sharing strategy, resulting in low GPU utilization. Tracking and analyzing the GPU in related technologies shows that 80% of the requested GPU utilization is less than 20%, and low utilization increases costs and prolongs job completion time, so improving utilization and reducing JCT is crucial.

[0021] The main reasons for low GPU utilization in related technologies are two-fold: first, resource allocation redundancy, users often plan more resources than they actually use, and about 88.78% of jobs use less than 50% of the requested GPU resources; the configuration file of the DLT job can cause GPU demand redundancy, and the actual utilization rate is much lower than the value shown in the configuration file; second, DLT jobs exhibit dynamic usage patterns during different training stages, resulting in fluctuating resource demands, and GPU idle periods for different configurations and models are significantly reduced, further reducing overall utilization.

[0022] GPU sharing techniques in related technologies fall into two categories: the first category divides GPUs into micro-instances before job deployment, but this can lead to resource fragmentation, and the optimization goal is to minimize fragmentation; the second category packs multiple jobs into one GPU, aiming to maximize resource utilization, but is limited by platform resource allocation restrictions, such as the Kubernetes scheduling platform's restrictions on GPU allocation, resulting in resource waste.

[0023] In view of this, the embodiments of the present application provide a graphics processor excess task allocation method, as shown in Figure 1 , which comprises: S1, acquire a plurality of to-be-allocated tasks (i.e., DLTs) and resource consumption data of each to-be-allocated task.

[0024] S2, divide the plurality of to-be-allocated tasks into task stages according to the resource consumption data of each to-be-allocated task, and generate an interleaved packing scheme for at least two to-be-allocated tasks based on the task stages, wherein the interleaved packing scheme comprises causing one processor to simultaneously process at least two to-be-allocated tasks.

[0025] Specifically, since the resource consumption of one task is different in different stages, packing the task based on the task stage can cause the processor to stagger the processing of the task, thereby providing efficiency.

[0026] S3, based on the interleaved packing scheme and according to the arrival time of the to-be-allocated tasks, allocate processors (i.e., GPUs) to the plurality of to-be-allocated tasks by a blocking scheduling mode and a non-blocking scheduling mode, wherein at least one processor is allocated with to-be-allocated tasks that exceed the total resource capacity of the processor.

[0027] Specifically, the blocking scheduling mode means that, for a certain to-be-allocated task, the allocation of the processor for the to-be-allocated task will not be preempted by other to-be-allocated tasks, and thus the to-be-allocated task will not be starved. The non-blocking scheduling mode is the opposite, and when a certain to-be-allocated task cannot be allocated a processor for a short time, the non-blocking scheduling mode will preferentially allocate a processor to other to-be-allocated tasks, thereby avoiding the head-of-line blocking (HOL) problem. The above method of the embodiment of the present application can combine the advantages of the two scheduling modes by using different scheduling modes in different situations.

[0028] Over-allocation, i.e., the above-mentioned at least one processor is allocated with to-be-allocated tasks that exceed the total resource capacity of the processor, combines the total GPU demand of a plurality of DLT jobs to exceed the target GPU capacity, and runs them on the target GPU. In specific implementation, over-allocation can be realized by pre-occupying processor resources, and theoretically can realize 100% utilization of GPU resources.

[0029] Embodiments of the present application provide a GPU overbooking scheduler, hereinafter referred to as HyperWeave, for performing the above-mentioned graphics processor overbooking method. First, embodiments of the present application provide a new complementary matching algorithm (i.e. the above-mentioned S2) for packing jobs to optimize GPU sharing in the time dimension, which identifies tasks with complementary resource requirements and binds them together to alleviate resource competition in the time dimension. Second, embodiments of the present application introduce a new multi-GPU orchestration algorithm (i.e. the above-mentioned S3), which dynamically allocates GPUs to the tasks to be allocated according to their arrival times, combining the blocking and non-blocking scheduling modes to avoid head-of-line blocking and stragglers in the spatial dimension. At the same time, embodiments of the present application allow overbooking of processors, i.e. the above-mentioned at least one processor is allocated more tasks to be allocated than the sum of the resource capacities of the processor, combined with the above-mentioned task alternation execution mechanism, by allowing tasks to temporarily overlap their resource requirements to improve GPU utilization. In related technologies, overbooking is essentially a free-for-all competition for resources among tasks, and severe resource competition will lead to an increase in job completion time. Embodiments of the present application avoid the head-of-line blocking problem through the above-mentioned multi-GPU orchestration algorithm, and can also prevent tasks from being starved.

[0030] Specifically, we define the workload factor ω as the sum of the GPU requirements (or the sum of the utilizations) of all tasks on a single GPU. When the workload factor does not exceed 100% (i.e. ω≤100%), the allocation strategy at this time is called low commitment; when the workload factor exceeds 100% (i.e. ω>100%), overbooking occurs. The two allocation strategies exhibit different characteristics in optimizing resource utilization.

[0031] The low commitment strategy involves imposing a limit on the resource usage of each task. This method prevents any single task from consuming all available GPU resources by guaranteeing a minimum resource level for each task, thereby ensuring the feasibility of running all tasks. In addition, due to the predictability of resource allocation, this method also brings predictable performance to the GPU. The above-mentioned limitation of the low commitment strategy is crucial in terms of responsiveness and reliability, however, the limited resource sharing often leads to low resource utilization efficiency, and fixed resource allocation can prevent some tasks from fully utilizing the GPU, resulting in resource waste and longer execution time.

[0032] The over-allocation strategy allows multiple tasks to freely compete for GPU resources without forced resource limits, which brings several benefits. First, this dynamic resource utilization enables tasks to adjust resource requests in real time according to their computational needs, thus more efficiently utilizing available GPU cores and memory. This preventive mechanism can significantly reduce idle resources, thus reducing the execution time of tasks. In addition, when tasks flexibly request resources, GPU parallel processing tasks can improve the throughput of the entire system, especially in the case of different computational loads in different models.

[0033] However, the fierce resource competition caused by over-allocation can make it difficult for each task to obtain sufficient resources, thus increasing the task completion time. Specifically, the simultaneous use of GPU memory increases the fragmentation level of tasks and reduces the available GPU memory, forcing the GPU to enter a low workspace, slow speed algorithm. Embodiments of the present application solve the problems caused by the over-allocation strategy while retaining the advantages of the over-allocation strategy in improving GPU resource utilization through the above complementary matching algorithm and multi-GPU orchestration algorithm.

[0034] Specifically, in some embodiments, the resource consumption data includes the resource consumption of the to-be-assigned task at different times. The step of dividing the task phase of the to-be-assigned task according to the resource consumption data of the to-be-assigned task in S2 includes: According to the resource consumption of the to-be-assigned task at different times, the task phase of the to-be-assigned task is divided into a cross-training period phase and a training period within phase, wherein for at least one resource, the resource consumption of the training period within phase is greater than the resource consumption of the cross-training period phase.

[0035] The interleaved packaging scheme in S2 includes: Make one processor process one to-be-assigned task in the cross-training period phase and another to-be-assigned task in the training period within phase at the same time.

[0036] The method provided in the above embodiments reduces the peak resource usage in the time dimension to avoid excessive resource competition, specifically by overlapping the peaks and troughs of resource usage for two tasks to be assigned. To more effectively stagger the peaks and troughs of resource usage for two tasks to be assigned, this embodiment divides the training process into an initial stage, a cross-training-cycle stage (hereinafter referred to as the cross-cycle stage), and a training-cycle-intra-stage stage (hereinafter referred to as the intra-cycle stage). The training cycle refers to the training cycle of DLT, where one training cycle represents the process of a deep learning model completing one learning and parameter update on the entire training dataset. A training cycle is relatively long, and dividing the process into stages according to the training cycle can further improve marginal returns. The DLT task in the cross-cycle stage includes data loading and checkpointing tasks, while the intra-cycle stage includes forward, backward, and gradient synchronization tasks. Different stages have different resource requirements, leading to fluctuations in resource usage, such as... Figure 2 As shown, Figure 2 The red waveform in the image represents the resource consumption of a certain DLT at different stages.

[0037] The above embodiments of the present invention allow some tasks to utilize GPU resources released by other tasks by interleaving two tasks to be assigned in cross-cycle and intra-cycle phases, thereby improving GPU utilization and reducing peak fluctuations in GPU utilization.

[0038] Specifically, the related technologies do not employ an interleaved packaging scheme, but only an over-allocation strategy for GPU usage, as illustrated in the diagram below. Figure 3 As shown, in GPU multiprocessor sharing mode, the basic oversubscription strategy causes four tasks (task 1, task 2, task 3, and task 4, with resource utilization rates of 60%, 50%, 70%, and 60%, respectively) to compete for GPU resources, resulting in a maximum GPU workload of up to 240%. This disordered competition slows down GPU processing speed and increases job completion time. A schematic diagram illustrating GPU usage using the oversubscription strategy of the interleaved packing scheme in this embodiment of the invention is shown below. Figure 4 As shown, over-optimization based on interleaved packing technology can effectively alleviate resource contention between tasks, with the GPU's maximum workload reduced to only 130%, and under ideal conditions, resource contention can be reduced by 50%. For example, without interleaved packing, when N tasks share the same GPU, at most N tasks may simultaneously contend for resources; however, with interleaved packing, multiple tasks form N / 2 pairs, so at any given time, at most only N / 2 tasks are in a resource contention state. The method provided in this embodiment of the invention distributes tasks in an orderly and alternating manner, reducing cross-task contention and improving GPU efficiency.

[0039] In practical implementation, Multi-Process Service (MPS) in related technologies can be used to enable GPUs to execute interleaved and packaged tasks in parallel on an over-allocated basis. This service facilitates GPU sharing among multiple processes. MPS allows multiple processes to share a single GPU context, eliminating the need for each process to have a separate context, thus significantly reducing context switching overhead. Furthermore, MPS uses a fair time-slicing method to allocate GPU computing resources to multiple tasks and determines the execution cycle of each task. This means that each task receives a specific allocation of computing time. This time-slicing scheduling allows processes packaged from multiple DLT tasks to pre-utilize GPU resources, thus laying the foundation for over-allocation.

[0040] Specifically, in some embodiments, the above method further includes: The processor can simultaneously process one unassigned task that is in a phase across training cycles and another unassigned task that is in a phase within a training cycle. The processor includes a phase lock, which is used to prevent the processor from processing two or more unassigned tasks that are in the same task phase at the same time.

[0041] In specific implementation, the above methods are as follows: Figure 5 and Figure 6 As shown, in most cases, Task A and Task B will run in phases (intra-cycle phases and cross-cycle phases), and the phase that is currently running will be locked, while the phase that has been completed will be unlocked, such as... Figure 5 As shown; when a task (task B) completes a phase first, it will be blocked because other tasks (task A) have not yet released the lock on another phase, as... Figure 6 As shown.

[0042] In practical implementation, to determine how to interleave and package multiple tasks to be assigned, it is necessary to model the resource consumption and time of each DLT task. This embodiment of the invention uses a quintuple. To describe a DLT task, in the formula, Indicates task The number of training cycles, Indicates training task The number of parallel operations, i.e., the number of GPUs. Representing tasks Resource consumption in the three phases (initial phase, cross-cycle phase, and intra-cycle phase). In each phase, average CPU utilization C, maximum memory usage M, average GPU utilization G, maximum GPU memory usage GM, and required time T can be calculated. This formula represents...

[0043] The embodiment of the application also formulates an over-allocation constraint. For each GPU, an over-allocation rate is defined , which sets the maximum workload factor of the device. The GPU workload is quantified by its peak GPU utilization. Therefore, the DLT tasks co-scheduled on the same GPU must satisfy the over-allocation constraint of the formula , where represents the set of tasks running simultaneously on the same GPU, is the GPU workload, is the average GPU usage of the three stages (initial stage, cross-period stage, and intra-period stage) of respectively. Thus, the allocation of CPU and memory remains at 100%, and the over-allocation rate alleviates low utilization through user-specified resource redundancy and inflated configuration file GPU requirements.

[0044] In some embodiments, the process of generating an interleaved packing scheme for at least two to-be-allocated tasks based on task stages in S2 described above includes: For at least two to-be-allocated tasks, calculate the resource complementarity value between the to-be-allocated tasks, and add the packing scheme of the two to-be-allocated tasks with the highest resource complementarity value to the interleaved packing scheme. The formula for calculating the resource complementarity value includes: ; ; wherein is the resource complementarity value of the to-be-allocated task and the to-be-allocated task , the greater the resource complementarity between the two tasks, is used to represent a plurality of computing indicators of the to-be-allocated task , is used to represent a plurality of computing indicators of the to-be-allocated task , is a complementarity value calculation function, represents the maximum value of and , , , , , the plurality of computing indicators of the to-be-allocated task and the to-be-allocated task both include the number of training periods , for two complementary tasks, it is best to be as close as possible in to avoid one task completing quickly in fewer periods and the other task running alone, resulting in idle resources. The to-be-allocated tasks a plurality of computing indicators of the task to be allocated Further, the plurality of computing indicators of the task to be allocated across the stages of the training cycle, the above a plurality of computing indicators of the task to be allocated across the stages of the training cycle, the above a plurality of computing indicators of the task to be allocated across the stages of the training cycle, the above a plurality of computing indicators of the task to be allocated across the stages of the training cycle, the above a plurality of computing indicators of the task to be allocated across the stages of the training cycle, the sum of the resource requirements of the two complementary tasks in the remaining five dimensions (C, M, G, GM and T) in different interleaving stages should be as close as possible, which helps to reduce resource fluctuations.

[0045] In some embodiments, in order to avoid prolonging the JCT of the DLT task with low GPU utilization fluctuation, the embodiments of the present application set a matching factor for the tasks participating in the matching process. Before the above step of generating an interleaving packing scheme for at least two tasks to be allocated based on the stages of the tasks, further comprising: calculating a matching factor for each stage of each task to be allocated. In response to the matching factor of the task to be allocated being greater than or equal to a preset matching factor threshold, an interleaving packing scheme is generated for the task to be allocated; or, the matching factor of the task to be allocated is less than the matching factor threshold, and a processor allocation scheme is generated for the task to be allocated separately.

[0046] Specifically, since the fluctuation of GPU resources is mainly caused by different tasks executed in different stages, the embodiments of the present application determine the above matching factor according to the duration of the tasks in different stages. The calculation formula of the matching factor includes: ; wherein, is the matching factor, is the duration of the task to be allocated across the stages of the training cycle, is the duration of the task to be allocated within the stages of the training cycle.

[0047] In specific implementation, the above matching factor threshold The value ranges from 0 to 0.5, with 0.16 being preferred.

[0048] This invention also provides pseudocode for the above-described complementary matching algorithm, as shown in Algorithm 1 below, wherein lines 1-5 of the pseudocode are used to base the algorithm on the above-described matching factor. Tasks that do not need to be packaged are removed, and the remaining tasks are packaged according to the number of GPUs. p Grouping is performed, and the code in lines 6 to 13 is based on the resource complementarity value. pair.

[0049] Algorithm 1: Complementary Matching Algorithm Input: Matching factor threshold The set of tasks to be assigned, S; 1: Initialize an empty set of unmatched tasks D and an empty list M; 2: Execute the following on task i in set S: 3: Calculate the matching factor for task i ; 4: If ,but: 5: Add task i to set D, where the key of task i is the GPU value; 6: Execute the following on the key in set D: 7: The set of unmatched keys S = D[key]; 8: Calculate the resource complementarity value for all tasks i and j in set S. ; 9: Put all ( Add task i and task j to list L; 10: According to Sort the contents of list L in descending order; 11: For the (in list L) Task i and Task j) are executed: 12: If task i and task j are not in list M, then: 13: Add (task i, task j) to list M; 14: Return to list M.

[0050] For the packaged tasks, when allocating processors, the aforementioned multi-GPU orchestration algorithm needs to consider both which tasks to prioritize for processor allocation and which processors to prioritize for each task. The following embodiments of the present invention will provide specific implementation methods for processor allocation. It should be noted that in the following embodiments, two interleaved packaged tasks are treated as one task and their corresponding processors are allocated accordingly.

[0051] In the related art, some schedulers employ first-in first-out (FIFO), Shortest Remaining Time First (SRTF), Shortest Remaining Service First (SRSF), and other strategies, but these strategies often encounter common queuing and head-of-line (HOL) blocking problems in the scheduling process. Here, HOL particularly refers to resource-related blocking, which means that tasks with small resource requirements can be blocked by tasks with large resource requirements. However, simply skipping larger tasks can cause these tasks to starve, thereby increasing the job completion time (JCT). This problem still exists under simple over-allocation.

[0052] Embodiments of the present application employ a bucket (equivalent to a queue)-based scheduling method, which employs a non-blocking scheduling method in the bucket, effectively alleviating HOL blocking. At the same time, between queues, blocking scheduling will still be employed to prevent starvation.

[0053] In some embodiments, the step of allocating processors to the plurality of tasks to be allocated by the blocking scheduling method and the non-blocking scheduling method described above comprises: According to the arrival time of the task to be allocated, at least two tasks to be allocated are classified into different time buckets. In specific implementation, the formula may be used, where Bid is the ID of the bucket into which the task is classified, the floor function is used to round down, is the task arrival time, is the bucket length.

[0054] The different time buckets are sorted in order of arrival time from early to late, and the tasks to be allocated are sorted in each time bucket in order of priority from high to low. Specifically, the priority can be obtained according to the SRSF described above, or it can be obtained by other means, and the embodiments of the present application do not limit it too much.

[0055] Each task to be allocated after sorting is allocated a processor in turn, wherein, in response to allocation of a processor within a time bucket by a non-blocking scheduling method, when a task to be allocated in front of the sorting cannot be allocated a processor, the task to be allocated is skipped; and in response to allocation of a processor between time buckets by a blocking scheduling method, when all tasks to be allocated in a time bucket in front of the sorting cannot be allocated a processor, allocation is suspended until all tasks to be allocated in the time bucket are allocated a processor.

[0056] Figure 7An application scenario of the above method in a specific embodiment is shown. In this scenario, the multi-GPU parallel tasks are designed to utilize only 40% of the capacity of each GPU. The tasks are grouped into two buckets according to the arrival time and prioritized using the SRSF method. In the GPU cluster, only 4 GPUs are available. Therefore, the highest priority task in bucket 1 (5, 1) requires 5 GPUs and cannot be executed. The second priority task in bucket 1 (2, 5) requires only 2 GPUs, so it is preferentially scheduled to be executed in the GPU of node b. At this time, there are 2 available GPUs left in the GPU cluster, while all other tasks in bucket 1 require more than 2 GPUs and are still blocked. Although there are enough GPUs in the cluster for the highest priority task in bucket 2 (1, 1), the task is also blocked because there are still unassigned tasks in bucket 1. This method prevents the complete HOL blocking caused by task (5, 1) while ensuring that task (5, 1) is not starved.

[0057] In the related art, for multi-GPU parallel tasks, the fragmentation problem of being allocated on different GPUs can also cause delay problems, because multi-GPU parallel tasks need to be synchronized between GPUs, and direct over-allocation cannot ensure that tasks obtain the same resources on all GPUs, and this difference can cause the execution speed to vary between different GPUs, causing faster GPUs to wait for slower GPUs, thereby significantly increasing JCT. Figure 8 The results of a typical greedy strategy for CPU selection in the related art are shown, that is, to preferentially allocate multi-GPU parallel tasks to idle GPUs. However, this method leads to uneven deployment, not only failing to accelerate parallel tasks, but also causing fragmentation of GPU resources, adversely affecting the execution of other tasks.

[0058] Therefore, in some embodiments, a plurality of processors are distributed in a plurality of nodes, each node including at least one processor; before allocating the plurality of to-be-allocated tasks to the processors by means of blocking scheduling and non-blocking scheduling, the selection of the GPU is further included, specifically comprising: Selecting part of the processors as available processors according to the resource remaining amount of each processor.

[0059] Sorting the plurality of nodes according to the number of available processors from high to low once, and sorting the nodes with the same number of available processors according to the total resource remaining amount of the available processors in the nodes from high to low twice.

[0060] In each node, sorting the plurality of available processors according to the resource remaining amount from high to low to obtain the sorting result of all available processors in all nodes.

[0061] Allocating processors for a plurality of to-be-allocated tasks by a blocking scheduling mode and a non-blocking scheduling mode, comprising: Allocating available processors for a plurality of to-be-allocated tasks according to a sorting result by a blocking scheduling mode and a non-blocking scheduling mode.

[0062] The method can allocate the multi-GPU parallel tasks to the GPUs with similar residual resource spaces when selecting processors, so as to alleviate the task fragmentation problem and reduce cross-server calculation. Figure 9 The method fully utilizes the GPU resources, prevents resource fragmentation, and alleviates the fragmentation problem. Figure 7 In the embodiment shown in

[0063] The embodiment of the application also provides a pseudo code of the multi-GPU orchestration algorithm, as shown in the following algorithm 2.

[0064] Algorithm 2: Multi-GPU orchestration algorithm.

[0065] Input: bucket length µ, DLT task bucket Q; 1: Initialize an empty GPU sorting list W; 2: Perform the following operations for task i in bucket Q: 3: Get the Bid value of task i (Bid is the ID value of the bucket, which is used to record the bucket that has not been scheduled); 4: Add task i to the bucket S corresponding to the Bid value; 5: Sort all buckets S in ascending order of Bid value; 6: Define bucket flag f=False, bucket ID b=None (f is a flag bit, which is used to record the first bucket that has not been scheduled); 7: Perform the following operations for task set S: 8: Get the matching list M of set S by using algorithm 1; 9: Sort the tasks in list M by using the SRSF algorithm; 10: For interleaved packed tasks Jm=(Jobi, Jobj), perform the following operations in list M: 11: On node x, get the number of Nx GPUs that meet the resource constraint conditions of Jm; 12: Arrange the nodes in descending order using Nx and the percentage of residual resources; 13: If , x∈all nodes, then: 14: Select top p GPUs according to node ranking; 15: Add selected GPUs to ranking list W; 16: Modify remaining resources of selected GPUs; 17: Else; 18: If f == False, then: 19: f = True, b = Bid (to record the ID of the bucket where the first scheduling failure occurs); 20: Else if b!= Bid, then: 21: Return ranking list W (stop scheduling if the current bucket and the bucket where the first scheduling failure occurs are not the same, so as to avoid the task in the first bucket being always unscheduled due to scheduling failure); 22: Return ranking list W.

[0066] The architecture of the HyperWeave scheduler provided by the embodiment of the application is shown in Figure 10 The HyperWeave scheduler is constructed based on a master-slave node mode, the master node is responsible for analyzing tasks, monitoring cluster resources and scheduling tasks, that is, the method S1 to S3 is executed, and the slave node is responsible for executing tasks and uploading the task completion status to the master node. Specifically, the master node includes three components, a task analyzer, a cluster monitor and a task scheduler, and the slave node includes a task executor component.

[0067] The task analyzer divides the initial stage, the cross-period stage and the in-period stage of the task by evaluating the time and resource consumption of different stages of the task to be allocated, that is, the method S1 is executed. The task analyzer executes the above method by running several batches, so as to record and infer the resource and time consumption of different stages.

[0068] The cluster monitor is used to track the resource information and task status in the cluster. The resource information includes the resource capacity and the remaining resources of each processor. The resource capacity is used to evaluate whether the interleaved packing scheme is feasible, and the remaining resources are used to identify the available worker nodes and available processors that can be used for task execution. After the task is completed, the cluster monitor records this event and updates the available resources for future scheduling.

[0069] The task scheduler uses a complementary matching algorithm to generate an interleaved packing scheme, and uses a multi-GPU orchestration algorithm to dynamically map the task to a suitable processor, that is, the methods S2 and S3 are executed.

[0070] The task executor executes the specified DLT task on the specified GPU, completes the packing of the task and parallel execution based on the interleaved packing scheme. The task executor needs to call the above multi-process service MPS to realize the context switching of the GPU.

[0071] The embodiments of this invention experimentally verified the data differences between the HyperWeave scheduler and other schedulers in related technologies (including FIFO, SRSF, MPS, SRTF, Muri, and Lucid) in terms of average task completion time, maximum task completion time, and task waiting time. Figures 11 to 13 As shown, where, Figure 11 The vertical axis represents the average task completion time. Figure 12 The vertical axis represents the maximum task completion time. Figure 13 The vertical axis represents the task waiting time. Figures 11 to 13 The x-axis represents different task traces. The results show that the method and scheduler provided in this embodiment of the invention reduce the average task completion time by 2.93 to 3.97 times, the maximum task completion time by 1.97 to 2.21 times, and the task waiting time by 5.76 to 8.66 times compared with other schedulers in related technologies.

[0072] Figure 14 The figure illustrates the GPU resource usage of the HyperWeave scheduler provided in this embodiment of the invention and other schedulers in related technologies (including FIFO, SRSF, MPS, SRTF, Muri, and Lucid) during task execution. The horizontal axis represents time, and the vertical axis represents GPU resource utilization. It can be clearly seen from the figure that HyperWeave is significantly better than other schedulers in terms of resource utilization.

[0073] Figure 15 The figure illustrates the GPU allocation of the HyperWeave scheduler provided in this embodiment of the invention and other schedulers (including FIFO, SRSF, MPS, SRTF, Muri, and Lucid) in related technologies before task execution. The horizontal axis represents time, and the vertical axis represents the GPU allocation rate. As shown in the figure, when ξ is set to 200%, HyperWeave achieves a maximum GPU allocation rate of nearly 200%, while the maximum GPU allocation rate of other schedulers is still 100%.

[0074] Figure 16 The figure illustrates the task waiting bucket length during task allocation for the HyperWeave scheduler provided in this embodiment of the invention and other schedulers in related technologies (including FIFO, SRSF, MPS, SRTF, Muri, and Lucid). The horizontal axis represents time, and the vertical axis represents the task waiting bucket length. As shown in the figure, in the HyperWeave scheduler, the bucket length decreases rapidly, indicating that the system has a strong job processing capability, which helps to reduce task waiting time.

[0075] Figure 17The figure illustrates the GPU allocation of the HyperWeave scheduler provided in this embodiment of the invention and other schedulers in related technologies (including FIFO, SRSF, MPS, SRTF, Muri, Lucid, and Tiresias) during actual task execution. The horizontal axis represents time, and the vertical axis represents the actual GPU allocation rate. As shown in the figure, HyperWeave achieves a maximum GPU allocation rate of nearly 200%, while the maximum GPU allocation rate of other schedulers is still 100%.

[0076] This invention also includes an experiment on the optimal parameter settings for the HyperWeave scheduler.

[0077] The embodiments of the present invention have achieved optimal over-allocation rates for different memory types and different GPU models. The experiment involved three GPUs: a P100 with 16GB of VRAM, a V100 with 32GB of VRAM, and an A100 with 80GB of VRAM. The results showed that the optimal over-allocation rate was [not specified in the original text]. The value is approximately 200%.

[0078] The embodiments of the present invention are also based on different matching factor thresholds. Simulation experiments were conducted. The value ranges from 0 to 0.5, adjusted in steps of 0.02. The experimental results show that when the matching factor threshold... When the JCT increases, it first decreases, then increases, and eventually stabilizes. In HyperWeave, although interleaving improves resource utilization, improper pairing of two jobs can also lead to an increase in JCT. Therefore, adjusting the matching factor threshold is crucial. However, it's crucial not to simply increase the value indiscriminately; an appropriate value must be chosen. After numerous experiments, a matching factor threshold was selected. As the optimal value.

[0079] The embodiments of the present invention are also based on different barrel lengths. A simulation experiment was conducted, and the length of the bucket was determined. The range is from 1×10 3 Up to 1×10 7 The experimental results show that as the bucket length increases, JCT and other performance metrics decrease. This phenomenon is easy to understand: a longer bucket length provides a wider range of non-blocking scheduling options, thereby reducing the GPU's idle percentage and improving overall efficiency. In practical applications, the bucket length needs to be determined based on specific requirements. When the bucket length is infinitely long, the job scheduling order in HyperWeave will be the same as SRSF. This embodiment of the invention uses a 1×10... 6As a balance value, to ensure that HyperWeave can achieve good system performance while preventing task starvation.

[0080] The embodiment of the application also performs an ablation experiment on the key technical solutions (complementary matching algorithm and multi-GPU scheduling algorithm). First, the experimental results of simultaneously applying the complementary matching algorithm and the multi-GPU scheduling algorithm are standardized (i.e., the results are all 1), and on this basis, the average task completion time is 1.1, the maximum task completion time is 1.16, and the task waiting time is 1.24 after replacing the multi-GPU scheduling algorithm with the basic scheduling strategy (FIFO) (keeping the complementary matching algorithm); after only using the multi-GPU scheduling algorithm, the average task completion time is 1.44, the maximum task completion time is 0.96, and the task waiting time is 1.74. As can be seen, it is the synergistic effect of the interleaved execution and scheduling technology provided by the embodiment of the application that enables HyperWeave to achieve better performance.

[0081] It should be noted that the method of the embodiment of the application can be executed by a single device, such as a computer or a server, etc. The method of the embodiment can also be applied in a distributed scenario, and completed by multiple devices cooperating with each other. In this distributed scenario, one of the multiple devices can only execute one or more steps in the method of the embodiment of the application, and the multiple devices will interact with each other to complete the above method.

[0082] It should be noted that some embodiments of the application have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in a different order than the order described above and still achieve desirable results. In addition, the processes depicted in the figures do not necessarily require the particular order shown or sequential order to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous.

[0083] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the embodiment of the application also provides a graphics processor task over-allocation device, as shown in Figure 18 , comprising: A task analysis module 10 is configured to obtain a plurality of to-be-allocated tasks and resource consumption data of each to-be-allocated task.

[0084] The task scheduling module 20 is configured to divide the plurality of to-be-allocated tasks into task stages according to resource consumption data of each to-be-allocated task, and generate an interleaving packaging scheme for at least two to-be-allocated tasks based on the task stages, wherein the interleaving packaging scheme comprises causing one processor to simultaneously process at least two to-be-allocated tasks. The task scheduling module 20 is further configured to allocate the plurality of to-be-allocated tasks to the processors by using a blocking scheduling mode and a non-blocking scheduling mode based on the interleaving packaging scheme and according to arrival times of the to-be-allocated tasks, wherein at least one processor is allocated with to-be-allocated tasks that exceed the total resource capacity of the processor.

[0085] The GPU over-task allocation scheduler, hereinafter referred to as HyperWeave, is used to execute the above-mentioned GPU over-task allocation method. First, the embodiment of the application provides a new complementary matching algorithm (executed by the task analysis module 10 and the task scheduling module 20) for packaging jobs to optimize GPU sharing in the time dimension. The algorithm identifies tasks with complementary resource requirements and binds them together to alleviate resource competition in the time dimension. Second, the embodiment of the application introduces a new multi-GPU orchestration algorithm (executed by the task scheduling module 20). The algorithm dynamically allocates GPUs to to-be-allocated tasks according to their arrival times, combining the blocking scheduling mode and the non-blocking scheduling mode to avoid head blocking and straggler problems in the spatial dimension. At the same time, the embodiment of the application allows over-allocation of tasks to processors, i.e., at least one processor is allocated with to-be-allocated tasks that exceed the total resource capacity of the processor. Combined with the above task alternation execution mechanism, the GPU utilization rate is improved by allowing tasks to temporarily overlap their resource requirements. In related technologies, over-allocation is essentially a free competition for resources among tasks, and severe resource competition will lead to an increase in job completion time. The embodiment of the application avoids the head blocking problem through the above multi-GPU orchestration algorithm, and can also prevent tasks from being starved.

[0086] The GPU over-task allocation device provided by the embodiment of the application can refer to Figure 10 The task analysis module 10 is the above-mentioned task analyzer, and the task scheduling module 20 is the above-mentioned task scheduler.

[0087] For the convenience of description, the above device is described as various modules in function. Of course, the functions of the modules can be implemented in one or more software and / or hardware when implementing the application.

[0088] The device of the above-mentioned embodiment is used to implement the corresponding GPU over-task allocation method in any of the above-mentioned embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be described here.

[0089] Figure 19A structural schematic diagram of an electronic device provided by an embodiment of the present application is shown.

[0090] As shown in Figure 19 , the electronic device can include a processor 1101 and a memory 1102 storing computer program instructions.

[0091] In particular, the processor 1101 can include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement one or more embodiments of the present application.

[0092] The memory 1102 can include a mass storage for information or instructions. By way of example, and not limitation, the memory 1102 can include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. Storage memory 1102 can be removable or non-removable (or fixed) as appropriate. Storage memory 1102 can be internal or external to the integrated gateway device as appropriate. In a particular embodiment, the memory 1102 is a non-volatile solid-state memory. In a particular embodiment, the memory 1102 includes read-only memory (ROM). The ROM can be mask programmed ROM, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), electrically alterable ROM (EAROM), or flash memory, or a combination of two or more of these, as appropriate.

[0093] The processor 1101 reads and executes the computer program instructions stored in the memory 1102 to perform the steps of the graphics processor over-task allocation method provided by the embodiments of the present application.

[0094] In one example, the electronic device can further include a transceiver 1103 and a bus 1104. As shown in Figure 19 , the processor 1101, the memory 1102 and the transceiver 1103 are connected through the bus 1104 and complete communication between each other.

[0095] Bus 1104 includes a hardware, software, or both. By way of example and not limitation, such bus can include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side BUS (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or another suitable bus or a combination of two or more of these. Where appropriate, bus 1104 can include one or more buses. Although the present embodiment describes and shows a particular bus, the present embodiment contemplates any suitable bus or interconnect.

[0096] The following is an embodiment of a computer readable storage medium provided by the present embodiment, which belongs to the same inventive concept as the above-mentioned embodiments of the graphics processor excess task allocation method. Details not described in the embodiment of the computer readable storage medium can be referred to the above-mentioned embodiments of the graphics processor excess task allocation method.

[0097] The present embodiment provides a storage medium containing computer executable instructions, which when executed by a computer processor, are used to perform a graphics processor excess task allocation method.

[0098] Of course, the computer executable instructions of the storage medium provided by the present embodiment are not limited to the above-mentioned method operations, and can also perform related operations in the graphics processor excess task allocation method provided by any embodiment of the present embodiment.

[0099] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer cloud platform (which can be a personal computer, server, or network cloud platform, etc.) to execute the graphics processor overload allocation method provided in the various embodiments of the present invention.

[0100] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the aforementioned element.

[0101] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments described above, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for allocating excess tasks to a graphics processing unit, characterized in that, The method comprises: obtaining a plurality of to-be-allocated tasks and resource consumption data of each of the to-be-allocated tasks; dividing task stages for the plurality of to-be-allocated tasks according to the resource consumption data of each of the to-be-allocated tasks, and generating an interleaved packing scheme for at least two of the to-be-allocated tasks based on the task stages, wherein the interleaved packing scheme comprises causing one of the processors to simultaneously process at least two of the to-be-allocated tasks; allocating processors to the plurality of to-be-allocated tasks by means of a blocking scheduling mode and a non-blocking scheduling mode based on the interleaved packing scheme and according to the arrival times of the to-be-allocated tasks, wherein at least one of the processors is allocated with to-be-allocated tasks that exceed the total resource capacity of the processor.

2. The method of claim 1, wherein, The resource consumption data comprises resource consumption amounts of a to-be-allocated task at different times; dividing task stages for a to-be-allocated task according to the resource consumption data of the to-be-allocated task comprises: dividing the task stages of a to-be-allocated task into a cross-training-period stage and a training-period-within stage according to the resource consumption amounts of the to-be-allocated task at different times, wherein for at least one resource, the resource consumption amount of the training-period-within stage is greater than the resource consumption amount of the cross-training-period stage; The interleaved packing scheme comprises: causing one of the processors to simultaneously process one to-be-allocated task in the cross-training-period stage and another to-be-allocated task in the training-period-within stage.

3. The method of claim 2, wherein the graphics processor over-tasking is performed by: The generating of the interleaved packing scheme for at least two of the to-be-allocated tasks based on the task stages comprises: for at least two of the to-be-allocated tasks, calculating resource complementarity values between the to-be-allocated tasks two by two, and adding the packing scheme of the two to-be-allocated tasks with the highest resource complementarity value to the interleaved packing scheme; The formula for calculating the resource complementarity value comprises: ； ； wherein, is a task to be assigned is a resource to be assigned is a complementary value of the resource to be assigned is a plurality of computing indicators for representing a task to be assigned is a plurality of computing indicators for representing a task to be assigned is a plurality of computing indicators for representing a task to be assigned is a plurality of computing indicators for representing a task to be assigned is a complementary value calculation function represents taking the maximum value in is a task to be assigned is a plurality of computing indicators for representing a task to be assigned The plurality of computing indicators of the task to be assigned and the plurality of computing indicators of the task to be assigned each include the number of training periods, the average CPU usage of the cross-training period stage, the maximum memory usage, the average GPU usage, the maximum GPU memory usage, the required time, and the average CPU usage, the maximum memory usage, the average GPU usage, the maximum GPU memory usage, and the duration of the stage within the training period.

4. The method of claim 2, wherein the step of Before the generating of the interleaved packing scheme for at least two of the to-be-allocated tasks based on the task stages, the method further comprises: calculating a matching factor according to the duration of each task stage of each to-be-allocated task; in response to the matching factor of a to-be-allocated task being greater than or equal to a preset matching factor threshold, generating the interleaved packing scheme for the to-be-allocated task, or in response to the matching factor of a to-be-allocated task being less than the matching factor threshold, generating a processor allocation scheme for the to-be-allocated task alone; The formula for calculating the matching factor comprises: ； wherein, the matching factor for the task to be assigned, the matching factor for the task to be assigned, the duration across training cycle phases for the task to be assigned, the duration across training cycle phases for the task to be assigned, the duration within training cycle phases for the task to be assigned, the duration within training cycle phases for the task to be assigned.

5. The method of claim 1, wherein the graphics processor over-tasking is performed by a graphics processing unit (GPU) driver. The allocating of the processors to the plurality of to-be-allocated tasks by means of the blocking scheduling mode and the non-blocking scheduling mode according to the arrival times of the to-be-allocated tasks comprises: grouping at least two to-be-allocated tasks into different time buckets according to the arrival times of the to-be-allocated tasks; sorting the different time buckets in order of arrival time from early to late, and sorting the to-be-allocated tasks in each of the time buckets in order of priority from high to low; allocating, for each of the sorted to-be-allocated tasks, a processor in turn, wherein, in response to the fact that, within the time bucket, a to-be-allocated task sorted in the front cannot be allocated a processor, the to-be-allocated task is skipped; and, in response to the fact that, between the time buckets, all to-be-allocated tasks in a time bucket sorted in the front cannot be allocated processors, the allocation is suspended until all to-be-allocated tasks in the time bucket are allocated processors.

6. The method of claim 1, wherein, The plurality of processors are distributed in a plurality of nodes, and each node includes at least one processor. Before the plurality of to-be-allocated tasks are allocated processors by the blocking scheduling mode and the non-blocking scheduling mode, the method further includes: selecting, according to the resource remaining amount of each processor, part of the processors as available processors; sorting, according to the number of available processors from high to low, the plurality of nodes once, and sorting, according to the total resource remaining amount of available processors in a node, the nodes with the same number of available processors twice from high to low; sorting, in each node, the plurality of available processors according to the resource remaining amount from high to low to obtain the sorting result of all available processors in all nodes; allocating, for the plurality of to-be-allocated tasks, the available processors according to the sorting result by the blocking scheduling mode and the non-blocking scheduling mode. allocating, for the plurality of to-be-allocated tasks, the available processors according to the sorting result by the blocking scheduling mode and the non-blocking scheduling mode.

7. The method of claim 2, further comprising: including: simultaneously processing, by the processor, one to-be-allocated task in the cross-training period phase and another to-be-allocated task in the training period phase, wherein the processor includes a phase lock, and the phase lock is used to prohibit the processor from simultaneously processing two or more to-be-allocated tasks in the same task phase.

8. A graphics processor overtasking apparatus, characterized by, including: a task analysis module configured to obtain a plurality of to-be-allocated tasks and resource consumption data of each to-be-allocated task; a task scheduling module configured to divide task phases for the plurality of to-be-allocated tasks according to the resource consumption data of each to-be-allocated task, and generate an interleaved packing scheme for at least two to-be-allocated tasks based on the task phases, wherein the interleaved packing scheme includes enabling one processor to simultaneously process at least two to-be-allocated tasks; the task scheduling module is further configured to allocate, for the plurality of to-be-allocated tasks, processors by the blocking scheduling mode and the non-blocking scheduling mode based on the interleaved packing scheme and according to the arrival time of the to-be-allocated tasks, wherein at least one processor is allocated to-be-allocated tasks exceeding the total resource capacity of the processor.

9. An electronic device, characterized by including: a processor; a memory configured to store executable instructions, wherein the processor is configured to read the executable instructions from the memory and execute the executable instructions to implement the method of any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the method of any one of claims 1 to 7.

Images