Large-scale priority scheduling method and device for GPU resources

Abstract

The embodiment of the application provides a large-specification priority scheduling method and device for GPU resources, dynamically adjusts the queue order based on the large task requirement and the resource specification demand satisfaction, performs the priority matching according to the large resource quantity, sequentially from large to small, performs the matching again from large to small after the matching is successful, preferentially allows the large task to completely occupy the large node, and concentrates the small task in the small node matched with the resource, so that the method can avoid the problems of the small task occupying the large resource and the large task running across nodes, prevents resource fragmentation, reduces the task cross-node communication cost and resource management overhead, and improves the resource utilization.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of data processing, in particular to a large-scale priority scheduling method and device for GPU resources. BACKGROUND

[0002] Kubernetes is an open-source container orchestration platform that automates the deployment, scaling, and management of containerized applications. In a Kubernetes cluster, you can run various containerized applications and manage these applications through various Kubernetes features. Currently, the scheduling of GPU computing resources for multiple Kubernetes clusters on the market is mainly implemented in three steps: queuing strategy, filtering stage, and scoring stage.

[0003] The queuing strategy of the scheduler is important for reasonably managing the Pods to be scheduled (the smallest deployable computing unit in Kubernetes) and ensuring scheduling efficiency and fairness. Common queuing strategies include priority sorting, fairness scheduling, and time sorting. Priority sorting sorts Pods according to their priority, with high-priority Pods being placed at the front of the queue and being scheduled first. Fairness scheduling ensures fairness between different users or different workloads. The scheduler uses a fair queuing strategy. Time sorting sorts Pods according to the time they enter the queue, following the first-come, first-served (FIFO) principle.

[0004] Common algorithms used in the filtering stage include resource checking algorithms, port checking algorithms, node affinity and anti-affinity algorithms, and Pod anti-affinity algorithms. Resource checking algorithms check whether the available resources (CPU, memory, GPU, storage, etc.) on the node meet the request requirements. The scheduler places nodes that meet the requirements in the candidate list. Port checking algorithms check whether the available ports on the node meet the request requirements. The scheduler places nodes that meet the requirements in the candidate list. Node affinity and anti-affinity algorithms filter nodes based on the node affinity and anti-affinity defined by the Pod, i.e., the Pod can only run on nodes with specific labels. Pod affinity and anti-affinity algorithms filter nodes based on the rules defined by the Pod, i.e., the Pod must run on the same node or different nodes as certain specific Pods.

[0005] The commonly used algorithm in the scoring stage is resource utilization algorithm, node affinity weight algorithm, LeastRequestedPriority algorithm and BalancedResourceAllocation algorithm. The resource utilization algorithm tends to schedule the Pod to the node with lower resource utilization to achieve balanced allocation of cluster resources. The node affinity weight algorithm scores the nodes according to the weight of the node affinity rule defined by the Pod. The LeastRequestedPriority algorithm selects the node with the least requested resources according to the usage of CPU and memory resources of the node. The calculation formula is: (cpu((capacity-requested)*10 / capacity)+memory((capacity-requested)*10 / capacity)) / 2. The BalancedResourceAllocation algorithm balances the usage of CPU and memory on the node to avoid the situation that the CPU usage of some nodes is too high while the memory usage is very low, or vice versa. The scheduler calculates the difference in the usage ratio of CPU and memory of the node, and the smaller the difference, the higher the score.

[0006] The existing scheduler strategy mainly tends to fairness and balance in the queuing strategy and the algorithm in the scoring stage, so as to ensure the performance and high reliability of the Pod running. However, for the GPU computing power operation scene, such as the requirement of 1-8 card GPU computing power on a single node, such as the uniform distribution of 1-4 card GPU Pod to the GPU node, the GPU computing power request of 4-8 cards cannot realize resource matching, thereby causing "computing power fragmentation" in computing power operation, and causing low overall utilization of computing power resources. SUMMARY

[0007] In view of the problems in the prior art, the application provides a large-specification priority scheduling method and device for GPU resources to improve the resource utilization of cluster nodes and avoid GPU resource fragmentation.

[0008] In order to solve at least one of the above problems, the application provides the following technical solutions:

[0009] In a first aspect, the application provides a large-specification priority scheduling method for GPU resources, comprising:

[0010] receiving a task submitted by a user, and queuing the task into a first queue according to priority;

[0011] obtaining a second queue by obtaining resource requirements of each task in the first queue and adjusting the order of the first queue based on the resource requirements of the tasks from large to small;

[0012] obtaining a third queue by obtaining resource availability of each node in the Kubernetes cluster and sorting each node in the third queue according to the resource availability from large to small;

[0013] performing multi-round matching between each task and each node in the third queue according to the order of the second queue, wherein each task in the first interval segment is matched from the first node in the third queue to the last node, and each task in the second interval segment is matched from the last node in the third queue to the first node, until all tasks are matched or resources are exhausted;

[0014] sending each task to each node for processing according to the matching result.

[0015] Further, the step of performing multi-round matching between each task and each node in the third queue according to the order of the second queue comprises:

[0016] if the matching is successful, removing the task from the second queue and updating the resource availability of the node;

[0017] if the matching fails, skipping the task.

[0018] Further, the step of performing multi-round matching between each task and each node in the third queue according to the order of the second queue comprises:

[0019] receiving a new task submitted by a user and inserting the new task into the first queue according to its priority to obtain a new first queue;

[0020] triggering the next round of matching after the current round of matching is completed, or restarting the matching process after the current scheduling period is completed;

[0021] obtaining a new second queue by obtaining resource requirements of each task in the new first queue and adjusting the order of the new first queue based on the resource requirements of the tasks from large to small;

[0022] performing multi-round matching between each task and each node in the third queue according to the order of the new second queue, until all tasks are matched or resources are exhausted.

[0023] Further, the step of performing multi-round matching of each of the tasks with each of the nodes in the third queue in the order of the second queue further comprises:

[0024] Reacquiring resource availability of each of the nodes in the Kubernetes cluster, and sorting each of the nodes in descending order of resource availability to obtain a new third queue;

[0025] Performing multi-round matching of each of the tasks with each of the nodes in the new third queue in the order of the second queue until all of the tasks are matched or resources are exhausted.

[0026] Further, the step of arranging the tasks into the first queue according to priority comprises:

[0027] Determining the type of the task submitted by the user, and determining the priority according to the type of the task; the type of the task includes an instant response task, a delay allowed task, and a moderate response time task;

[0028] Extracting time-related attributes of the task, and further determining the priority of each type of the task according to the time-related attributes; the time-related attributes include submission time, estimated execution duration, deadline, and service level agreement.

[0029] Further, the step of arranging the tasks into the first queue according to priority further comprises:

[0030] Multi-dimensional weight allocation is performed on the type of the task and the time-related attributes;

[0031] Setting conditions for recalculating the priority; the conditions include: fixed period, arrival of new task;

[0032] Setting preemption rules: the priority of a new task is higher than that of a running task, and the preempted task supports state saving;

[0033] Setting a time decay mechanism: gradually increasing the priority of a task that has been waiting for a long time.

[0034] Further, the step of obtaining resource requirements of each of the tasks in the first queue comprises:

[0035] Specifying a GPU model according to the characteristics of the task submitted by the user;

[0036] Determining the number of GPUs based on the parallelism of the task;

[0037] Calculating the resource requirements of the task from the GPU model and the number of GPUs.

[0038] In a second aspect, the application provides a large-scale priority scheduling device for GPU resources, comprising:

[0039] a task receiving module, configured to receive tasks submitted by a user and arrange the tasks into a first queue according to priorities;

[0040] a task adjusting module, configured to obtain resource requirements of the tasks in the first queue, adjust an order of the first queue based on the resource requirements of the tasks from large to small to obtain a second queue, and divide the second queue into a first interval segment and a second interval segment according to the resource requirements, and a resource requirement of each task in the first interval segment is greater than a resource requirement of each task in the second interval segment;

[0041] a node adjusting module, configured to obtain resource available amounts of nodes in a Kubernetes cluster, and sort the nodes according to the resource available amounts from large to small to obtain a third queue;

[0042] a matching module, configured to perform multi-round matching between the tasks and the nodes in the third queue according to the order of the second queue, match each task in the first interval segment from a first node in the third queue to a rear node, and match each task in the second interval segment from a last node in the third queue to a front node, until all the tasks are matched or resources are exhausted;

[0043] a scheduling module, configured to send the tasks to the nodes for processing according to a matching result.

[0044] In a third aspect, the present application provides an electronic device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the GPU resource-oriented large-specification priority scheduling method when executing the program.

[0045] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program, and the computer program implements the steps of the GPU resource-oriented large-specification priority scheduling method when executed by a processor.

[0046] In a fifth aspect, the present application provides a computer program product, which comprises a computer program / instruction, and the computer program / instruction implements the steps of the GPU resource-oriented large-specification priority scheduling method when executed by a processor.

[0047] According to the technical solution, the application provides a large-specification priority scheduling method and device for GPU resources, dynamically adjusts the queue order based on the large task requirement and the resource specification requirement, matches the resources in the order from large to small, and then matches the resources in the order from large to small again after the first matching is successful, so that large tasks are allowed to completely occupy large nodes and small tasks are allowed to be concentrated in small nodes, the problems of small tasks occupying large resources and large tasks running across nodes are avoided, resource fragmentation is prevented, the task cross-node communication cost and resource management overhead are reduced, and the resource utilization rate is improved. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0049] Figure 1 FIG. 1 is a flowchart of a large-specification priority scheduling method for GPU resources in an embodiment of the present application;

[0050] Figure 2 FIG. 2 is a flowchart of a large-specification priority scheduling method for GPU resources in an embodiment of the present application;

[0051] Figure 3 FIG. 3 is a structural diagram of a large-specification priority scheduling device for GPU resources in an embodiment of the present application;

[0052] Figure 4 FIG. 4 is a structural diagram of an electronic device in an embodiment of the present application.

[0053] Reference signs:

[0054] Electronic device 9600, central processor 9100, memory 9140, communication module 9110, input unit 9120, audio processor 9130, display 9160, power supply 9170, buffer memory 9141, application / function storage unit 9142, data storage unit 9143, driver program storage unit 9144, antenna 9111, speaker 9131, and microphone 9132. DETAILED DESCRIPTION

[0055] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0056] The acquisition, storage, use, and processing of data in this application all comply with the relevant provisions of national laws and regulations.

[0057] In view of the problems existing in the prior art, this application provides a method and apparatus for large-scale priority scheduling of GPU resources. Based on the large number of tasks that can meet the resource specification requirements, the queue order is dynamically adjusted. The task is matched in order of resource quantity, from large to small. After a successful match, the task is matched again from large to small. Large tasks are given priority to fully occupy large nodes, while small tasks are concentrated in small nodes with matching resources. This can avoid the problem of small tasks occupying large resources and large tasks running across nodes, prevent resource fragmentation, reduce the cost of cross-node communication and resource management overhead, and improve resource utilization.

[0058] To improve the resource utilization of cluster nodes and avoid GPU resource fragmentation, this application provides an embodiment of a large-scale priority scheduling method for GPU resources, see [link to relevant documentation]. Figure 1 and Figure 2 The large-scale priority scheduling method for GPU resources specifically includes the following:

[0059] Step S101: Receive tasks submitted by users and arrange the tasks into a first queue according to priority (i.e., Figure 2 (The enqueue in the queue).

[0060] Optionally, in this embodiment, the steps for determining priority based on task type and task time are as follows:

[0061] Step 1: Task Classification and Characteristic Analysis

[0062] 1. Classify tasks according to task type ( Figure 2 The task types in the algorithm include: real-time tasks, which require immediate response, such as online services and emergency calculations; batch or offline tasks, which allow for delays, such as data analysis and model training; and interactive tasks, which require a moderate response time, such as user queries and real-time monitoring.

[0063] 2. Extract time-related attributes of the task ( Figure 2Time-related attributes: submission time, estimated execution time, deadline, service level agreement (SLA).

[0064] Step 2: Building a priority evaluation model

[0065] Multi-dimensional weight distribution for task type and time-related attributes: for example, task type weight is 40%, real-time task > interactive task > batch processing task; deadline urgency weight is 30%, the closer to the deadline, the higher the priority; execution time weight is 20%, short task priority (reduce resource occupation time) or long task priority (avoid resource starvation); submission time weight is 10%, the earlier the task enters the system, the higher the priority.

[0066] Priority score = (task type weight × type coefficient) + (deadline weight × time urgency coefficient) + (execution time weight × time length coefficient) + (submission time weight × submission time coefficient)

[0067] Coefficient examples: type coefficient: real-time = 1.0, interactive = 0.7, batch processing = 0.3; time urgency coefficient: 1 - (remaining time / total allowed time) (the more urgent, the greater the value); time length coefficient: short task = 1.0 (SJF strategy) or long task = 0.8 (avoid starvation); submission time coefficient: 1 - (1 / submission time) (the longer the submission time, the greater the value).

[0068] Step 3: Dynamic priority adjustment strategy

[0069] 1. Real-time monitoring and re-evaluation: recalculate the priority of all unfinished tasks every fixed period (such as 30 seconds), or trigger local reordering immediately when a new task arrives.

[0070] 2. Preemption rules: conditions for preemption: new task priority higher than running task, and preoccupied task supports state saving (such as checkpointing). Preemption compensation: preoccupied task automatically promoted priority in the next cycle (anti-starvation).

[0071] 3. Time decay mechanism: gradually increase the priority of long-waiting tasks.

[0072] Formula: priority decay bonus = waiting time × decay factor (such as 0.1 / minute)

[0073] In this embodiment, by quantifying task types and time factors and combining them with a dynamic adjustment mechanism, a priority system that can both respond to urgent needs and maintain system efficiency can be constructed, finding a balance between real-time performance and fairness.

[0074] Optionally, in this embodiment, the tasks submitted by the user include online tasks and offline tasks. To obtain the optimal processing time for offline tasks and to rationally allocate resource supply, this embodiment limits the dequeue time for offline tasks.

[0075] Specifically, the method for determining the offline task dequeue time is as follows: collect the historical load values ​​of each node in the Kubernetes cluster over a recent period, and calculate the average load of all nodes at the same hour; calculate the cumulative load value of all nodes for a set time period after each hour based on the average load value; if the time period is insufficient, collect and accumulate the load values ​​of today; select the hour with the smallest cumulative load value as the offline task dequeue time, and the dequeue duration is equal to the set time period.

[0076] Optionally, in this embodiment, if the number of times the cumulative load value is the smallest is greater than 1, the cumulative load value is recalculated within a set number of times; if the number of times the cumulative load value is the smallest is still greater than 1 within a set number of times, the first time the cumulative load value is the smallest is used as the dequeue time for the offline task.

[0077] For example, refer to Figure 2 As shown, after receiving the tasks submitted by the user, the sorting algorithm creates a first queue based on task priority: Task 1, Task 2, and Task 3. The resource load prediction module in the sorting algorithm calculates the dequeue time for Task 3. The offline task strategy module collects the cluster load at each hour from 0:00 to 24:00 every day for the past 1-15 days, i.e., the load values ​​of all nodes in the Kubernetes cluster, and calculates the average load of all nodes at the same hour: [0.2, 0.1, 0.3, 0.5, 0.6, 0.2, 0.6, 0.2, 0.5, 0.8, 0.7, 0.6, 0.4, 0.2, 0.4, 0.6, 0.7, 0.8, 0.5, 0.4]. The values ​​are calculated as follows: [0.2, 0.5, 0.4, 0.2], and the cumulative load value for the next 8 hours is calculated sequentially. If it is less than 8 hours, today's load value is used for the cumulative load value. The resulting value is: [2.7, 3.0, 3.7, 4.1, 4.2, 4.0, 4.0, 3.8, 4.2, 4.4, 4.4, 4.2, 4.0, 3.8, 4.1, 4.1, 3.7, 3.2, 2.5, 2.3, 2.4, 2.8, 2.5, 2.7]. The hour containing the minimum value (if the minimum values ​​are the same, the cumulative load value is recalculated using the same rules. If the minimum values ​​are still the same after 3 iterations, the first minimum value is used) is the start time for submitting the offline task. Figure 2The off-line task time selection in the method) is 19:00, and lasts for 8 hours to 3:00 the next day, after which the submission of the off-line task is stopped.

[0078] In this embodiment, by the method, that is, by the mixed scheduling of the on-line task and the off-line task, the best time for submitting the off-line task is predicted through the historical time node utilization comprehensive measurement, which greatly improves the utilization rate of the cluster nodes, guarantees the large resource demand of the on-line user task, and takes into account the use experience of the on-line user.

[0079] Step S102: Obtain the resource demand of each task in the first queue, and adjust the order of the first queue based on the resource demand of the task from large to small to obtain a second queue (that is, the de-queue queue in the method) ; according to the resource demand, the second queue is divided into a first interval segment and a second interval segment, and the resource demand of each task in the first interval segment is greater than the resource demand of each task in the second interval segment. Figure 2

[0080] Optionally, in this embodiment, the operation process of obtaining the task resource demand (that is, the resource specification algorithm in the method) is as follows: Figure 2

[0081] I. Determine the GPU model required by the task

[0082] 1. Task characteristics and GPU model matching: for example, large model training (such as LLM) tasks, the recommended GPU model is NVIDIA A100 / H100; inference service (high throughput) tasks, the recommended GPU model is NVIDIA T4 / L4, etc.

[0083] 2. Method of specifying GPU model: explicit declaration: the user directly specifies the model when submitting the task (such as --gpu-type=a100); implicit matching: the system automatically recommends according to the task metadata.

[0084] II. Determine the number of GPUs

[0085] Based on the parallelism of the task: data parallelism requirement, required GPU number = ceil(total data volume / single card maximum batch size); model parallelism requirement, required GPU number = model layer number / number of layers per GPU.

[0086] III. Quantify the GPU resource demand

[0087] GPU resource demand = GPU model * number * 10000, 10000 is an expansion coefficient.

[0088] In this embodiment, by closely combining the hardware specifications and the task characteristics, accurate GPU resource demand evaluation and allocation can be achieved.​​

[0089] Step S103: Obtain the resource availability of each node in the Kubernetes cluster, and sort each node in descending order of resource availability to obtain a third queue.

[0090] In this embodiment, after reordering, the tasks in the second queue are dequeued and submitted to the scheduler, and the tasks are received by the task distributor in the scheduler. The node load module of the scheduler sorts each node in descending order of resource availability to obtain a third queue.

[0091] Optionally, in this embodiment, the total amount of GPU and the number of allocated GPU of all nodes can be viewed using the native kubectl command, and the resource availability of each node is the difference between the total amount of GPU and the number of allocated GPU of the node.

[0092] Step S104: Each task is matched with each node in the third queue in the order of the second queue. Each task in the first interval segment is matched from the first node in the third queue to the last node, and each task in the second interval segment is matched from the last node in the third queue to the first node, until all tasks are matched or resources are exhausted.

[0093] In this embodiment, the first queue is dynamically adjusted based on the task requirement and the resource specification requirement to form the second queue. The tasks in the second queue are matched with nodes in descending order of resource quantity, and after successful matching, the tasks are matched again in descending order of resource quantity, until all tasks are matched or resources are exhausted.

[0094] Specifically:

[0095] 1. First round of matching:

[0096] Sort all pending tasks in descending order of GPU demand (e.g., 8GPU task→2GPU task→1GPU task).

[0097] Sort available nodes in descending order of remaining GPU resources (e.g., Node1 remaining 8GPU→Node2 remaining 8GPU→Node3 remaining 4GPU).

[0098] Try to match the largest task to the largest available node:

[0099] If the matching is successful (e.g., 8GPU task is allocated to Node1), remove the task from the queue and update the node resource (Node1 remaining 0GPU).

[0100] If the match fails (e.g. the current max node only has 4 GPUs left, but the task requires 8 GPUs), skip this task.

[0101] 2. Second round of matching:

[0102] After completing the first round of matching, recheck the remaining resources and task queue:

[0103] The remaining node resources may have changed (e.g. Node1 is occupied, but Node2 still has 8 GPUs left).

[0104] There may still be unassigned tasks in the task queue (e.g. tasks skipped due to insufficient resources).

[0105] Repeat the first round of matching logic: again sort the remaining tasks from large to small, and the remaining nodes from large to small.

[0106] 3. Continue trying to match until all tasks are assigned or resources are exhausted.

[0107] Specific scenario example:

[0108] Example 1

[0109] Assume the initial state: task queue (sorted by GPU requirements): Task B (8 GPUs), Task C (2 GPUs), Task A (1 GPU).

[0110] Available nodes: Node1 (8 GPUs), Node2 (8 GPUs), Node3 (4 GPUs).

[0111] First round of matching:

[0112] Task B (8 GPUs) tries to match Node1 (8 GPUs) → succeeds, Node1 is full.

[0113] Remaining tasks: Task C (2 GPUs), Task A (1 GPU)

[0114] Remaining nodes: Node2 (8 GPUs), Node3 (4 GPUs)

[0115] Second round of matching (again from large to small):

[0116] Reorder remaining tasks: Task C (2 GPUs) → Task A (1 GPU)

[0117] Reorder remaining nodes: Node2 (8 GPUs) → Node3 (4 GPUs)

[0118] Task C (2 GPUs) tries to match Node3 (4 GPUs) → allocation succeeds, Node3 is left with 2 GPUs.

[0119] Remaining task: Task A (1 GPU)

[0120] Remaining node resource: Node2 (8 GPUs)

[0121] Final task A is assigned to the remaining 8 GPUs of Node2 (if sharing is allowed), or waits for new resources to be released.

[0122] In this embodiment, the design can prioritize tasks that require more GPUs, such as 8-GPU tasks, reducing the waiting time for large tasks and avoiding small tasks occupying large resources (such as running small tasks with 2 8-GPU nodes), and improving overall resource utilization.

[0123] Optionally, in this embodiment, the second queue is divided into a first interval segment and a second interval segment according to resource requirements, and the resource requirements of each task in the first interval segment are greater than the resource requirements of each task in the second interval segment; each task in the first interval segment is matched from the first node in the third queue, i.e., the node with the largest available resources, and each task in the second interval segment is matched from the last node in the third queue, i.e., the node with the smallest available resources.

[0124] For example, the first interval segment is [5-8] GPU requirements, and the second interval segment is [1-4] GPU requirements.

[0125] As in Example 1, Task B (8 GPUs) is matched from Node1 (8 GPUs) backward, and Task C (2 GPUs) and Task A (1 GPU) are matched from Node3 (4 GPUs) forward.

[0126] In this embodiment, the design is used to place small tasks on small nodes to reserve large nodes for large tasks, further avoiding small tasks occupying large resources, preventing resource fragmentation, and improving resource utilization.

[0127] Optionally, in this embodiment, when a new task appears after the first round of matching, the system will dynamically handle it through the following mechanisms:

[0128] 1. New task triggers immediate queue update

[0129] Insertion into queue: The new task is inserted into the task queue according to its priority (e.g., online tasks have high priority and offline tasks have low priority).

[0130] Example:

[0131] If the new task is online task D (4 GPUs, high priority) → inserted at the head of the queue

[0132] If the new task is offline task E (3 GPUs, low priority) → inserted at the tail of the queue

[0133] 2. Dynamically interrupt the current scheduling period

[0134] Strategy selection:

[0135] The processing manner of the high-priority task is: after the completion of the current round of matching, the next round of matching is triggered.

[0136] The processing manner of the low-priority task is: after the completion of the current period, the matching process is restarted.

[0137] Operation logic:

[0138] If the new task is a high-priority task (such as an online service), the system will suspend the task allocation that is currently being performed (the tasks that have not been submitted for scheduling remain in the state), add the new task to the queue and reorder it (task D→task B→task C→task A→task E), and immediately start a new round of "matching from large to small".

[0139] If the new task is a low-priority task (such as offline computing), the system will add it to the tail of the queue, continue to complete the resource allocation of the current scheduling period, and process the new task in the next scheduling period.

[0140] In this embodiment, the resource availability of each node in the Kubernetes cluster is updated in real time with the occupation and release of each node, and each node is sorted in the order of resource availability from large to small.

[0141] Step S105: According to the matching result, each task is sent to each node for processing.

[0142] As can be seen from the above description, the large-specification priority scheduling method for GPU resources provided in the embodiments of the present application dynamically adjusts the queue order based on the large task requirement and the ability to meet the resource specification requirement, and matches in priority according to the resource amount, from large to small in turn, and then matches from large to small again after the matching is successful, so that large tasks can completely occupy large nodes and small tasks can be concentrated in small nodes with matched resources, which can avoid the problem of small tasks occupying large resources and large tasks running across nodes, prevent resource fragmentation, reduce task cross-node communication cost and resource management overhead, and improve resource utilization.

[0143] In order to improve the resource utilization of the cluster nodes and avoid GPU resource fragmentation, the present application provides an embodiment of a large-specification priority scheduling device for GPU resources for implementing all or part of the contents of the large-specification priority scheduling method for GPU resources, as shown in Figure 3 The large-specification priority scheduling device for GPU resources specifically includes the following contents:

[0144] The task receiving module 10 is configured to receive tasks submitted by a user and arrange the tasks into a first queue according to priorities.

[0145] The task adjusting module 20 is configured to obtain resource requirements of the tasks in the first queue, adjust the order of the first queue based on the resource requirements of the tasks from large to small to obtain a second queue, and divide the second queue into a first interval segment and a second interval segment according to the resource requirements, wherein the resource requirements of the tasks in the first interval segment are greater than the resource requirements of the tasks in the second interval segment.

[0146] The node adjusting module 30 is configured to obtain resource availability of nodes in a Kubernetes cluster, and sort the nodes according to the resource availability from large to small to obtain a third queue.

[0147] The matching module 40 is configured to perform multi-round matching between the tasks and the nodes in the third queue according to the order of the second queue, wherein the tasks in the first interval segment are matched from the first node in the third queue to the rear, and the tasks in the second interval segment are matched from the last node in the third queue to the front, until all the tasks are matched or the resources are exhausted.

[0148] The scheduling module 50 is configured to send the tasks to the nodes for processing according to the matching results.

[0149] As can be seen from the above description, the GPU resource-oriented large-specification priority scheduling device provided by the embodiments of the present application can dynamically adjust the queue order based on the large task requirement and the ability to meet the resource specification requirement, preferentially match according to the resource quantity, and sequentially match from large to small. After successful matching, the large task is preferentially allowed to completely occupy the large node, and the small task is concentrated in the small node with matched resources. The device can avoid the problems of small task occupying large resources and large task running across nodes, prevent resource fragmentation, reduce task cross-node communication cost and resource management overhead, and improve resource utilization.

[0150] From the hardware level, in order to improve the resource utilization of the cluster nodes and avoid GPU resource fragmentation, the present application provides an embodiment of an electronic device for implementing all or part of the contents of the GPU resource-oriented large-specification priority scheduling method, which specifically includes the following contents:

[0151] The processor, the memory, the communications interface and the bus; wherein the processor, the memory, the communications interface complete the communication between each other through the bus; the communications interface is used for realizing the information transmission between the large-scale priority scheduling device facing the GPU resource and the core business system, the user terminal and the related database and other related equipment; the logic controller can be a desktop computer, a tablet computer and a mobile terminal and the like, and the embodiment is not limited thereto. In the embodiment, the logic controller can refer to the embodiment of the large-scale priority scheduling method facing the GPU resource and the embodiment of the large-scale priority scheduling device facing the GPU resource, the contents of which are incorporated herein, and the repeated parts will not be described herein.

[0152] It can be understood that the user terminal can include a smart phone, a tablet electronic device, a network set-top box, a portable computer, a desktop computer, a personal digital assistant (PDA), a vehicle-mounted device, a smart wearable device and the like. The smart wearable device can include smart glasses, a smart watch, a smart bracelet and the like.

[0153] In actual application, part of the large-scale priority scheduling method facing the GPU resource can be executed on the electronic device as described above, or all operations can be completed in the client device. Specifically, the selection can be made according to the processing capacity of the client device and the limitation of the user use scenario, and the like. The present application is not limited thereto. If all operations are completed in the client device, the client device can further include a processor.

[0154] The client device described above can have a communication module (i.e. a communication unit) and can be communicatively connected with a remote server to realize the data transmission with the server. The server can include a server of the task scheduling center side, and the server of the intermediate platform can also be included in other implementation scenarios, for example, the server of the third-party server platform communicatively connected with the server of the task scheduling center. The server can include a single computer device, a server cluster composed of multiple servers or a distributed server structure.

[0155] Figure 4 A schematic block diagram of the system configuration of the electronic device 9600 of the embodiment of the present application is shown in FIG. 9. As shown in the figure, the electronic device 9600 can include a central processor 9100 and a memory 9140; the memory 9140 is coupled to the central processor 9100. It is worth noting that the structure shown in the figure is exemplary; other types of structures can also be used to supplement or replace the structure to realize the telecommunication function or other functions. Figure 4 Figure 4 The structure shown in the figure is exemplary; other types of structures can also be used to supplement or replace the structure to realize the telecommunication function or other functions. ​

[0156] In an embodiment, the GPU resource-oriented large-specification priority scheduling method function can be integrated into the central processor 9100. The central processor 9100 can be configured to control as follows:

[0157] Step S101: receiving a task submitted by a user, and arranging the task into a first queue according to priority;

[0158] Step S102: obtaining resource requirements of each task in the first queue, and adjusting the order of the first queue based on the resource requirements of the tasks from large to small to obtain a second queue; dividing the second queue into a first interval segment and a second interval segment according to resource requirements, and the resource requirements of each task in the first interval segment being greater than the resource requirements of each task in the second interval segment;

[0159] Step S103: obtaining resource availability of each node in the Kubernetes cluster, and sorting each node in the third queue according to the resource availability from large to small to obtain a third queue;

[0160] Step S104: each task is matched with each node in the third queue according to the order of the second queue, each task in the first interval segment is matched from the first node in the third queue to the rear, and each task in the second interval segment is matched from the last node in the third queue to the front, until all the tasks are matched or the resources are exhausted;

[0161] Step S105: sending each task to each node for processing according to the matching result.

[0162] As can be seen from the above description, the electronic device provided by the embodiments of the present application dynamically adjusts the queue order based on the large task requirement and the ability to meet the resource specification requirement, preferentially matches according to the resource amount, and sequentially matches from large to small. After successful matching, it is matched from large to small again, preferentially allows large tasks to occupy large nodes completely, and small tasks are concentrated in small nodes for resource matching, which can avoid the problem of small tasks occupying large resources and large tasks running across nodes, prevent resource fragmentation, reduce task cross-node communication cost and resource management overhead, and improve resource utilization.

[0163] In another embodiment, the GPU resource-oriented large-specification priority scheduling device can be configured separately from the central processor 9100, for example, the GPU resource-oriented large-specification priority scheduling device can be configured as a chip connected with the central processor 9100, and the GPU resource-oriented large-specification priority scheduling method function is realized through the control of the central processor.

[0164] As Figure 4As shown, the electronic device 9600 may further include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is worth noting that the electronic device 9600 does not necessarily need to include these components. Figure 4 All components shown; in addition, the electronic device 9600 may also include Figure 4 For components not shown, please refer to existing technologies.

[0165] like Figure 4 As shown, the central processing unit 9100, sometimes also referred to as a controller or operating control, may include a microprocessor or other processor device and / or logic device, which receives inputs and controls the operation of various components of the electronic device 9600.

[0166] The memory 9140 may be, for example, one or more of a cache, flash memory, hard drive, removable media, volatile memory, non-volatile memory, or other suitable devices. It may store the aforementioned failure-related information, and also store a program for executing that information. The central processing unit 9100 may execute the program stored in the memory 9140 to perform information storage or processing, etc.

[0167] Input unit 9120 provides input to central processing unit 9100. Input unit 9120 may be, for example, a keypad or touch input device. Power supply 9170 provides power to electronic device 9600. Display 9160 displays images and text. Display may be, for example, an LCD display, but is not limited thereto.

[0168] The memory 9140 can be a solid-state memory, such as a read-only memory (ROM), random access memory (RAM), a SIM card, etc. It can also be a memory that retains information even when power is off, can be selectively erased, and contains more data; examples of this type of memory are sometimes referred to as EPROMs. The memory 9140 can also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application / function storage unit 9142 for storing application programs and function programs or processes for executing the operation of the electronic device 9600 via the central processing unit 9100.

[0169] The memory 9140 can further include a data storage 9143 for storing data such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. A driver storage 9144 of the memory 9140 can include various drivers of the electronic device for communication functions and / or for performing other functions of the electronic device (e.g., a messaging application, a phonebook application, etc.).

[0170] The communication module 9110 is a transmitter / receiver that transmits and receives signals via the antenna 9111. The communication module 9110 (transmitter / receiver) is coupled to the central processor 9100 to provide input signals and receive output signals, as in the case of a conventional mobile communication terminal.

[0171] Based on different communication technologies, a plurality of communication modules 9110 can be provided in the same electronic device, such as a cellular network module, a Bluetooth module, and / or a wireless local area network module, etc. The communication module 9110 (transmitter / receiver) is further coupled to the speaker 9131 and the microphone 9132 via the audio processor 9130 to provide audio output via the speaker 9131 and receive audio input from the microphone 9132, thereby implementing a conventional telecommunication function. The audio processor 9130 can include any suitable buffer, decoder, amplifier, etc. In addition, the audio processor 9130 is coupled to the central processor 9100, thereby enabling recording on the local device via the microphone 9132 and enabling playing of a sound stored on the local device via the speaker 9131.

[0172] The embodiment of the present application further provides a computer readable storage medium capable of implementing all steps of the GPU resource-oriented large-scale priority scheduling method in which the execution subject is a server or a client in the above-mentioned embodiment, wherein a computer program is stored on the computer readable storage medium, and the computer program is executed by a processor to implement all steps of the GPU resource-oriented large-scale priority scheduling method in which the execution subject is a server or a client in the above-mentioned embodiment, for example, the processor executes the computer program to implement the following steps:

[0173] Step S101: receiving a task submitted by a user, and arranging the task into a first queue according to a priority;

[0174] Step S102: obtaining resource requirements of each task in the first queue, and adjusting an order of the first queue based on the resource requirements of the tasks from large to small to obtain a second queue; dividing the second queue into a first interval segment and a second interval segment according to the resource requirements, and a resource requirement amount of each task in the first interval segment is greater than a resource requirement amount of each task in the second interval segment;

[0175] Step S103: Obtain the resource available amount of each node in the Kubernetes cluster, and sort each node in descending order of resource available amount to obtain a third queue;

[0176] Step S104: Each task is matched with each node in the third queue in the order of the second queue. Each task in the first interval segment is matched from the first node in the third queue to the rear, and each task in the second interval segment is matched from the last node in the third queue to the front, until all tasks are matched or resources are exhausted;

[0177] Step S105: According to the matching result, each task is sent to each node for processing.

[0178] From the above description, the computer readable storage medium provided by the embodiments of the application dynamically adjusts the queue order based on the large task requirement and the resource specification requirement, preferentially matches according to the resource amount, and sequentially matches from large to small. After successful matching, it is matched from large to small again, preferentially allows large tasks to occupy large nodes, and small tasks are concentrated in small nodes with matched resources, which can avoid the problem of small tasks occupying large resources and large tasks running across nodes, prevent resource fragmentation, reduce task cross-node communication cost and resource management overhead, and improve resource utilization.

[0179] The embodiments of the application also provide a computer program product capable of implementing all steps of the large-scale priority scheduling method for GPU resources in the above-mentioned embodiments. The computer program / instructions are executed by the processor to implement the steps of the large-scale priority scheduling method for GPU resources, for example, the computer program / instructions implement the following steps:

[0180] Step S101: Receive a task submitted by a user, and arrange the task into a first queue according to priority;

[0181] Step S102: Obtain the resource requirement of each task in the first queue, and adjust the order of the first queue from large to small based on the resource requirement of the task to obtain a second queue; divide the second queue into a first interval segment and a second interval segment according to resource requirement, and the resource requirement of each task in the first interval segment is greater than the resource requirement of each task in the second interval segment;

[0182] Step S103: Obtain the resource available amount of each node in the Kubernetes cluster, and sort each node in descending order of resource available amount to obtain a third queue;

[0183] Step S104: each of the tasks is matched with each of the nodes in the third queue in the order of the second queue, each of the tasks in the first interval section is matched from the first node in the third queue backward, each of the tasks in the second interval section is matched from the last node in the third queue forward, until all the tasks are matched or the resources are exhausted;

[0184] Step S105: each of the tasks is sent to each of the nodes for processing according to the matching result.

[0185] From the above description, it can be seen that the computer program product provided by the embodiment of the present application dynamically adjusts the queue order based on the large task requirement and the resource specification requirement, preferentially matches the resources in large quantity, and sequentially matches from large to small. After successful matching, it is matched from large to small again, preferentially allows large tasks to completely occupy large nodes, and concentrates small tasks in small nodes of resource matching, which can avoid problems such as small tasks occupying large resources and large tasks running across nodes, prevents resource fragmentation, reduces task cross-node communication cost and resource management overhead, and improves resource utilization.

[0186] Those skilled in the art will understand that the embodiments of the present application can be provided as a method, device, or computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0187] The present application is described with reference to flowcharts and / or block diagrams of methods, devices (apparatuses), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The device that implements the functions specified in one or more flows and / or blocks. Figure 1 The device that implements the functions specified in one or more flows and / or blocks.

[0188] These computer program instructions can also be stored in a computer readable storage medium that can guide the computer or other programmable data processing apparatus to work in a specific way, so that the instructions stored in the computer readable storage medium produce a product including instruction devices, which implement the functions specified in the flowcharts and / or block diagrams. Figure 1one or more processes and / or blocks Figure 1 the function specified in the one or more blocks.

[0189] These computer program instructions can also be loaded into computer or other programmable data processing devices, so that a series of operation steps are performed on the computer or other programmable data processing devices to generate a computer implemented process, so that the instructions executed on the computer or other programmable data processing devices provide a process for implementing the flow Figure 1 one or more processes and / or blocks Figure 1 the function specified in the one or more blocks.

[0190] The principles and implementation manners of the present application are described in the specific embodiments in the present application, and the above embodiment description is only for helping to understand the method of the present application and its core idea; meanwhile, for the general skilled in the art, according to the idea of the present application, the specific implementation manners and application ranges will have changes, and the above description should not be understood as the limitation of the present application.

Claims

1. A large-scale priority scheduling method for GPU resources, characterized in that, The method comprises: receiving user-submitted tasks and arranging the tasks into a first queue according to priority; the user-submitted tasks include online tasks and offline tasks; the determination method of the offline task dequeue time is: collecting the historical load values of each node in the Kubernetes cluster in a recent period of time, and calculating the load average of all nodes at the same whole point; calculating the load cumulative value of each whole point after a set time period of all nodes according to the load average, and if the time length is insufficient, the load values of today are collected and accumulated; selecting the whole point with the smallest load cumulative value as the offline task dequeue time, and the dequeue time length is equal to the set time period; if the number of whole points with the smallest load cumulative value is greater than 1, the load cumulative value is recalculated within a set number of times; if the number of whole points with the smallest load cumulative value is still greater than 1 within a set number of times, the first whole point with the smallest load cumulative value is used as the offline task dequeue time; obtaining the resource requirements of each task in the first queue, and adjusting the order of the first queue based on the resource requirements of the tasks from large to small to obtain a second queue; the second queue is divided into a first interval segment and a second interval segment according to the resource requirements, and the resource requirement amount of each task in the first interval segment is greater than the resource requirement amount of each task in the second interval segment; obtaining the resource available amount of each node in the Kubernetes cluster, and sorting each node in the third queue according to the order of the resource available amount from large to small to obtain a third queue; each task is matched with each node in the third queue in multiple rounds according to the order of the second queue, each task in the first interval segment is matched from the first node in the third queue to the rear, and each task in the second interval segment is matched from the last node in the third queue to the front, until all tasks are matched or resources are exhausted; sending each task to each node for processing according to the matching result.

2. The large-scale priority scheduling method for GPU resources according to claim 1, wherein, The step of matching each task with each node in the third queue in multiple rounds according to the order of the second queue comprises: if the matching is successful, the task is removed from the second queue, and the resource available amount of the node is updated; if the matching fails, the task is skipped.

3. The GPU resource oriented large-scale priority scheduling method according to claim 1, wherein, The step of matching each task with each node in the third queue in multiple rounds according to the order of the second queue further comprises: receiving a new task submitted by a user and inserting the new task into the first queue according to its priority to obtain a new first queue; after the current round of matching is completed, triggering the next round of matching, or after the current scheduling period is completed, starting the matching process again; obtaining the resource requirements of each task in the new first queue, and adjusting the order of the new first queue based on the resource requirements of the tasks from large to small to obtain a new second queue; each task is matched with each node in the third queue in multiple rounds according to the order of the new second queue, until all tasks are matched or resources are exhausted.

4. The GPU resource oriented large-scale priority scheduling method according to claim 1, wherein, The step of matching each task with each node in the third queue in multiple rounds according to the order of the second queue further comprises: Reacquire resource availability of each node in the Kubernetes cluster, and sort each node in descending order of resource availability to obtain a new third queue; Each of the tasks is matched with each of the nodes in the new third queue in the order of the second queue for multiple rounds until all the tasks are matched or the resources are exhausted.

5. The GPU resource oriented large-scale priority scheduling method according to claim 1, wherein, The step of arranging the tasks into the first queue according to the priority comprises: Determining the type of the task submitted by the user, and determining the priority according to the type of the task; the type of the task includes a task requiring immediate response, a task allowing delay, and a task requiring moderate response time; Extracting time-related attributes of the task, and further determining the priority of each type of the task according to the time-related attributes; the time-related attributes include submission time, estimated execution time, deadline, and service level agreement.

6. The GPU resource oriented large-scale priority scheduling method according to claim 1, wherein, The step of arranging the tasks into the first queue according to the priority further comprises: Multi-dimensional weight allocation is performed on the type of the task and the time-related attributes; Setting conditions for recalculating the priority; the conditions include: fixed period, new task arrival; Setting preemption rules: the priority of a new task is higher than that of a running task, and the preempted task supports a saved state; Setting a time decay mechanism: gradually increasing the priority of a task waiting for a long time.

7. The GPU resource oriented large-scale priority scheduling method according to claim 1, wherein, The step of obtaining the resource requirement of each of the tasks in the first queue comprises: Specifying a GPU model according to the characteristics of the task submitted by the user; Determining the number of GPUs based on the parallelism of the task; Calculating the resource requirement of the task from the GPU model and the number of GPUs.

8. A large-scale priority scheduling apparatus for GPU resources, characterized by, The device comprises: A task receiving module for receiving a task submitted by a user and arranging the task into a first queue according to a priority; the task submitted by the user includes an online task and an offline task; the determination method of the offline task out-of-queue time is: collecting historical load values of each node in a Kubernetes cluster in a recent period of time, and calculating load average values of all nodes at the same whole point; calculating load cumulative values of all nodes at a set time period after each whole point according to the load average values, and if the time length is insufficient, the load values of today are collected and added; selecting the whole point with the smallest load cumulative value as the out-of-queue time of the offline task, and the out-of-queue time length is equal to the set time period; if the number of whole points with the smallest load cumulative value is greater than 1, the load cumulative values are recalculated within a set number of times; if the number of whole points with the smallest load cumulative value is still greater than 1 within the set number of times, the first whole point with the smallest load cumulative value is used as the out-of-queue time of the offline task; A task adjusting module for obtaining resource requirements of each of the tasks in the first queue, and adjusting the order of the first queue in descending order of the resource requirements to obtain a second queue; the second queue is divided into a first interval segment and a second interval segment according to the resource requirements, and the resource requirement of each of the tasks in the first interval segment is greater than the resource requirement of each of the tasks in the second interval segment; a node adjustment module, configured to obtain resource available amounts of nodes in a Kubernetes cluster, and sort the nodes according to the resource available amounts in descending order to obtain a third queue; a matching module, configured to perform multi-round matching of the tasks with the nodes in the third queue according to the order of the second queue, wherein the tasks in the first interval segment are matched from the first node in the third queue to the rear, the tasks in the second interval segment are matched from the last node in the third queue to the front, and the matching is performed until all the tasks are matched or resources are exhausted; a scheduling module, configured to send the tasks to the nodes for processing according to the matching result.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the steps of the GPU resource-oriented large-specification priority scheduling method in any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the GPU resource-oriented large-specification priority scheduling method in any one of claims 1 to 7.

Images