Large-specification priority scheduling method and device for GPU (Graphics Processing Unit) resources

Abstract

The embodiment of the invention provides a large-specification priority scheduling method and device for GPU (Graphics Processing Unit) resources, which are characterized in that a queue sequence is dynamically adjusted on the basis of a large task demand quantity and capability of meeting resource specification requirements, the priority matching is performed according to a large resource quantity, the sequence is from large to small, and the matching is performed again from large to small after successful matching, so that a large task completely occupies a large node preferentially; according to the method, the problems that the small tasks occupy large resources and the large tasks run across the nodes can be avoided, resource fragmentation is prevented, the task cross-node communication cost and resource management overhead are reduced, and the resource utilization rate is increased.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a large-scale priority scheduling method and device for GPU resources. Background Art

[0002] Kubernetes is an open-source container orchestration platform for automating the deployment, scaling, and management of containerized applications. Within a Kubernetes cluster, you can run a variety of containerized applications and manage them through Kubernetes' various features. Currently, scheduling GPU computing resources across multiple Kubernetes clusters is typically implemented in three steps: queuing strategy, filtering, and scoring.

[0003] The scheduler's queuing strategy is crucial for properly managing scheduled pods (the smallest deployable computing unit in Kubernetes) and ensuring scheduling efficiency and fairness. Common queuing strategies include priority sorting, fair scheduling, and time sorting. Priority sorting sorts pods by their priority, with higher-priority pods placed at the front of the queue and scheduled first. Fair scheduling ensures fairness among different users or workloads, and the scheduler employs a fair queuing strategy. Time sorting sorts pods by the time they enter the queue, using the first-come, first-served (FIFO) principle.

[0004] The commonly used algorithms in the filtering stage are the resource check algorithm, the port check algorithm, the node affinity and anti-affinity algorithm, and the Pod (the smallest deployable computing unit in Kubernetes) anti-affinity algorithm. The resource check algorithm mainly checks whether the available resources (CPU, memory, GPU, storage, etc.) on the node meet the request requirements. The scheduler will put the nodes that meet the requirements into the candidate list. The port check algorithm mainly checks whether the available ports on the node meet the request requirements. The scheduler will put the nodes that meet the requirements into the candidate list. The node affinity and anti-affinity algorithms filter nodes according to the node affinity and anti-affinity defined by the Pod, that is, the Pod can only run on nodes with specific labels. The Pod affinity and anti-affinity algorithms are based on the rules defined by the Pod, that is, the Pod must run on the same node or on different nodes with certain specific Pods.

[0005] Commonly used algorithms during the scoring phase include the resource utilization algorithm, the node affinity weight algorithm, the LeastRequestedPriority algorithm, and the BalancedResourceAllocation algorithm. The resource utilization algorithm tends to schedule pods to nodes with lower resource utilization to achieve balanced distribution of cluster resources. The node affinity weight algorithm scores nodes based on the weights of the node affinity rules defined by the pod. The LeastRequestedPriority algorithm selects the node with the least requested resources based on the node's CPU and memory usage. The calculation formula is: (cpu((capacity-requested)*10 / capacity)+memory((capacity-requested)*10 / capacity)) / 2. The BalancedResourceAllocation algorithm balances CPU and memory usage on nodes to avoid situations where some nodes have excessive CPU usage and low memory usage, or vice versa. The scheduler calculates the difference in the CPU and memory usage ratios of the nodes; the smaller the difference, the higher the score.

[0006] Existing scheduler strategies prioritize fairness and balance in their queuing and scoring algorithms, ensuring Pod performance and high reliability. However, in GPU computing power operation scenarios, if a single node needs to meet the computing power requirements of 1-8 GPU cards, and if Pods with 1-4 GPU cards are evenly distributed across GPU nodes, then computing power requests for 4-8 GPU cards will not be matched with resources. This results in "computing power fragmentation" in computing power operations and low overall utilization of computing resources. Summary of the Invention

[0007] In response to the problems in the prior art, the present application provides a large-scale priority scheduling method and device for GPU resources to improve the resource utilization of cluster nodes and avoid GPU resource fragmentation.

[0008] In order to solve at least one of the above problems, the present application provides the following technical solutions:

[0009] In a first aspect, the present application provides a large-scale priority scheduling method for GPU resources, comprising:

[0010] Receive tasks submitted by users and arrange the tasks into a first queue according to priority;

[0011] Obtaining resource requirements of each of the tasks in the first queue, and adjusting the order of the first queue from largest to smallest based on the resource requirements of the tasks to obtain a second queue; dividing the second queue into a first interval and a second interval according to the resource requirements, wherein the resource requirements of each of the tasks in the first interval are greater than the resource requirements of each of the tasks in the second interval;

[0012] Obtain the available resources of each node in the Kubernetes cluster, and sort the nodes in descending order of available resources to obtain a third queue;

[0013] Each task is matched with each node in the third queue in multiple rounds according to the order of the second queue. Each task in the first interval is matched from the first node in the third queue backward, and each task in the second interval is matched from the last node in the third queue forward, until all tasks are matched or resources are exhausted.

[0014] According to the matching results, each of the tasks is sent to each of the nodes for processing.

[0015] Furthermore, the step of performing multiple rounds of matching of each of the tasks with each of the nodes in the third queue in the order of the second queue includes:

[0016] If the match is successful, the task is removed from the second queue and the available resources of the node are updated;

[0017] If the match fails, the task is skipped.

[0018] Furthermore, the step of performing multiple rounds of matching of each of the tasks with each of the nodes in the third queue in the order of the second queue further includes:

[0019] Receive a new task submitted by a user, and insert the new task into the first queue according to its priority, to obtain a new first queue;

[0020] After the current round of matching is completed, the next round of matching is triggered, or after the current scheduling cycle is completed, the matching process is restarted;

[0021] Obtaining resource requirements of each of the tasks in the new first queue, and adjusting the order of the new first queue from largest to smallest based on the resource requirements of the tasks to obtain a new second queue;

[0022] Each of the tasks is matched with each of the nodes in the third queue in the order of the new second queue for multiple rounds until all the tasks are matched or the resources are exhausted.

[0023] Furthermore, the step of performing multiple rounds of matching of each of the tasks with each of the nodes in the third queue in the order of the second queue further includes:

[0024] Re-obtain the available resources of each node in the Kubernetes cluster, and sort the nodes in descending order of available resources to obtain a new third queue;

[0025] Each of the tasks is matched with each of the nodes in the new third queue in the order of the second queue for multiple rounds until all the tasks are matched or the resources are exhausted.

[0026] Furthermore, the step of arranging the tasks into a first queue according to priority includes:

[0027] Determining the type of the task submitted by the user and determining a priority based on the task type; the task types include tasks requiring immediate response, tasks that allow for delay, and tasks requiring a moderate response time;

[0028] Extracting time-related attributes of the tasks, and further determining the priorities of the tasks of each type based on the time-related attributes; the time-related attributes include submission time, estimated execution time, deadline, and service level agreement.

[0029] Furthermore, the step of arranging the tasks into a first queue according to priority also includes:

[0030] Performing multi-dimensional weight assignment on the task type and the time-related attributes;

[0031] Setting the conditions for recalculating the priority; the conditions include: fixed period, new task arrival;

[0032] Set preemption rules: new tasks have higher priority than running tasks, and preempted tasks support state saving;

[0033] Set up a time decay mechanism: gradually increase the priority of tasks that have been waiting for a long time.

[0034] Furthermore, the step of obtaining the resource requirements of each of the tasks in the first queue includes:

[0035] Specifying a GPU model according to the characteristics of the task submitted by the user;

[0036] Determining the number of GPUs based on the parallelism of the task;

[0037] The resource requirement of the task is calculated based on the GPU model and the number of GPUs.

[0038] In a second aspect, the present application provides a large-scale priority scheduling device for GPU resources, comprising:

[0039] A task receiving module is used to receive tasks submitted by users and arrange the tasks into a first queue according to priority;

[0040] a task adjustment module, configured to obtain resource requirements of each of the tasks in the first queue, and adjust the order of the first queue from largest to smallest based on the resource requirements of the tasks to obtain a second queue; divide the second queue into a first interval and a second interval according to the resource requirements, wherein the resource requirements of each of the tasks in the first interval are greater than the resource requirements of each of the tasks in the second interval;

[0041] A node adjustment module is used to obtain the available resources of each node in the Kubernetes cluster and sort the nodes in descending order of available resources to obtain a third queue;

[0042] a matching module configured to perform multiple rounds of matching of each task with each node in the third queue in the order of the second queue, wherein each task in the first interval is matched from the first node in the third queue backward, and each task in the second interval is matched from the last node in the third queue forward, until all tasks are matched or resources are exhausted;

[0043] The scheduling module is used to send each of the tasks to each of the nodes for processing according to the matching results.

[0044] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and runnable on the processor, wherein when the processor executes the program, the steps of the large-scale priority scheduling method for GPU resources are implemented.

[0045] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the large-scale priority scheduling method for GPU resources.

[0046] In a fifth aspect, the present application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the large-scale priority scheduling method for GPU resources.

[0047] It can be seen from the above technical solution that the present application provides a large-scale priority scheduling method and device for GPU resources. The queue order is dynamically adjusted based on the task demand being large and meeting the resource specification requirements. Priority is given to matching tasks with large resource amounts, from large to small. After a successful match, the tasks are matched again from large to small. Large tasks are given priority to completely occupy large nodes, and small tasks are concentrated in small nodes where resources are matched. This can avoid the problem of small tasks occupying large resources and large tasks running across nodes, prevent resource fragmentation, reduce the communication cost of tasks across nodes and resource management overhead, and improve resource utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0049] Figure 1 This is a flow chart of a large-scale priority scheduling method for GPU resources in an embodiment of the present application;

[0050] Figure 2 This is a second flow chart of a large-scale priority scheduling method for GPU resources in an embodiment of the present application;

[0051] Figure 3 This is a structural diagram of a large-scale priority scheduling device for GPU resources in an embodiment of the present application;

[0052] Figure 4 Schematic diagram of the structure of the electronic device in the embodiment of the present application.

[0053] Reference numerals:

[0054] Electronic device 9600, central processing unit 9100, memory 9140, communication module 9110, input unit 9120, audio processor 9130, display 9160, power supply 9170, buffer memory 9141, application / function storage unit 9142, data storage unit 9143, driver program storage unit 9144, antenna 9111, speaker 9131, microphone 9132. DETAILED DESCRIPTION

[0055] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0056] The acquisition, storage, use, and processing of data in this application's technical solution comply with relevant national laws and regulations.

[0057] Taking into account the problems existing in the prior art, the present application provides a large-scale priority scheduling method and device for GPU resources. The queue order is dynamically adjusted based on the task demand being large and meeting the resource specification requirements. Priority is given to matching tasks with large resource quantities, from large to small. After a successful match, the tasks are matched again from large to small. Large tasks are prioritized to completely occupy large nodes, and small tasks are concentrated in small nodes where resources are matched. This can avoid the problem of small tasks occupying large resources and large tasks running across nodes, prevent resource fragmentation, reduce the communication cost of tasks across nodes and resource management overhead, and improve resource utilization.

[0058] In order to improve the resource utilization of cluster nodes and avoid GPU resource fragmentation, this application provides an embodiment of a large-scale priority scheduling method for GPU resources. Figure 1 and Figure 2 The large-scale priority scheduling method for GPU resources specifically includes the following contents:

[0059] Step S101: Receive tasks submitted by users and arrange the tasks into the first queue (i.e. Figure 2 in the enqueue queue).

[0060] Optionally, in this embodiment, the steps of determining the priority according to the task type and task time are as follows:

[0061] Step 1: Task classification and feature analysis

[0062] 1. Classify tasks by task type ( Figure 2 The task type algorithm in the algorithm includes: real-time tasks: requiring immediate response, such as online services and urgent computing; batch tasks or offline tasks: allowing delays, such as data analysis and model training; interactive tasks: requiring moderate response time, such as user queries and real-time monitoring.

[0063] 2. Extracting time-related attributes of tasks ( Figure 2The time algorithm in the system includes: submission time: the time when the task enters the system; estimated execution time: the time required to complete the task; deadline: the latest time when the task must be completed; service level agreement (SLA): the maximum allowed delay or completion time.

[0064] Step 2: Build a priority assessment model

[0065] Multi-dimensional weights are assigned to task types and time-related attributes. For example, the weight of task type is 40%, with real-time tasks > interactive tasks > batch processing tasks; the weight of deadline urgency is 30%, with the closer the deadline, the higher the priority; the weight of execution time is 20%, with short tasks taking priority (reducing resource occupation time) or long tasks taking priority (avoiding resource starvation); the weight of submission time is 10%, with the earlier the task enters the system, the higher the priority.

[0066] Priority score = (task type weight × type coefficient) + (deadline weight × time urgency coefficient) + (execution time weight × duration coefficient) + (submission time weight × submission time coefficient)

[0067] Coefficient examples: Type coefficient: real-time = 1.0, interactive = 0.7, batch = 0.3; Time urgency coefficient: 1-(remaining time / total allowed time) (the more urgent the value, the larger the value); Duration coefficient: short task = 1.0 (SJF strategy) or long task = 0.8 (avoid starvation); Submission time coefficient: 1-(1 / submission time) (the longer the submission time, the larger the value).

[0068] Step 3: Dynamic priority adjustment strategy

[0069] 1. Real-time monitoring and re-evaluation: Recalculate the priorities of all unfinished tasks at fixed intervals (e.g., 30 seconds), or immediately trigger local reordering when new tasks arrive.

[0070] 2. Preemption rules: Preemption is allowed only when the new task has a higher priority than the currently running task and the preempted task supports state preservation (e.g., checkpointing). Preemption compensation: The preempted task automatically has its priority increased in the next cycle (to prevent starvation).

[0071] 3. Time decay mechanism: gradually increase the priority of tasks that have been waiting for a long time.

[0072] Formula: Priority decay bonus = waiting time × decay factor (such as 0.1 / minute)

[0073] In this embodiment, by quantifying task types and time factors and combining them with a dynamic adjustment mechanism, a priority system can be constructed that can respond to urgent needs while maintaining system efficiency, finding a balance between real-time performance and fairness.

[0074] Optionally, in this embodiment, the tasks submitted by the user include online tasks and offline tasks. In order to obtain the best processing time for offline tasks and reasonably allocate the supply of resources, this embodiment limits the dequeue time of offline tasks.

[0075] Specifically, the offline task dequeue time is determined as follows: the historical load values of each node in the Kubernetes cluster over the recent period are collected, and the load average value of all nodes at the same hour is calculated; the cumulative load value of all nodes in a set time period after each hour is calculated based on the load average value. If the time period is insufficient, the cumulative load value of today is collected; the hour with the smallest cumulative load value is selected as the offline task dequeue time, and the dequeue time is equal to the set time period.

[0076] Optionally, in this embodiment, if the number of hourly times when the load cumulative value is the smallest is greater than 1, the load cumulative value is recalculated within a set number of times; if the number of hourly times when the load cumulative value is the smallest is still greater than 1 within the set number of times, the first hourly time when the load cumulative value is the smallest is used as the dequeue time of the offline task.

[0077] Exemplary, reference Figure 2 As shown in the figure, after receiving the tasks submitted by the user, the sorting algorithm sorts the first queue based on the task priority: Task 1, Task 2, Task 3, and calculates the queue discharge time of Task 3 based on the resource load prediction module in the sorting algorithm; the offline task strategy module collects the cluster load at each hour from 0:00 to 24:00 every day in the past 1-15 days, that is, the load value of all nodes in the Kubernetes cluster, and calculates the load average of all nodes at the same hour: [0.2, 0.1, 0.3, 0.5, 0.6, 0.2, 0.6, 0.2, 0.5, 0.8, 0.7, 0.6, 0.4, 0.2, 0.4, 0.6, 0.7, 0.8, 0.5, 0.4 , 0.2, 0.5, 0.4, 0.2], and calculate the load accumulation value for 8 hours after each hour. If it is less than 8 hours, use today's load accumulation value to get the value: [2.7, 3.0, 3.7, 4.1, 4.2, 4.0, 4.0, 3.8, 4.2, 4.4, 4.4, 4.2, 4.0, 3.8, 4.1, 4.1, 3.7, 3.2, 2.5, 2.3, 2.4, 2.8, 2.5, 2.7], select the minimum value (if the minimum values are the same, recalculate the load accumulation value according to the same rule. If the minimum values are the same after 3 cycles, use the first minimum value). The hour where it is located is the start time for submitting the offline task ( Figure 2In this example, the offline task time is 19:00, which lasts for 8 hours until 3:00 the next day, after which offline task submission stops.

[0078] In this embodiment, through this method, that is, through the mixed scheduling of online tasks and offline tasks, the optimal time for offline task submission is predicted through comprehensive measurement of historical node utilization, which greatly improves the utilization of cluster nodes, ensures the large resource requirements of online user tasks, and takes into account the user experience of online users.

[0079] Step S102: Obtain the resource requirements of each task in the first queue, and adjust the order of the first queue from large to small based on the resource requirements of the tasks to obtain the second queue (i.e. Figure 2 The second queue is divided into a first interval segment and a second interval segment according to resource requirements, and the resource requirement of each task in the first interval segment is greater than the resource requirement of each task in the second interval segment.

[0080] Optionally, in this embodiment, obtaining the task resource requirements (i.e. Figure 2 The operation process of the resource specification algorithm in is as follows:

[0081] 1. Determine the GPU model required for the task

[0082] 1. Matching task characteristics with GPU models: For example, for large model training tasks (such as LLM), the recommended GPU model is NVIDIA A100 / H100; for inference service (high throughput) tasks, the recommended GPU model is NVIDIA T4 / L4, etc.

[0083] 2. Methods for specifying GPU models: Explicit declaration: The user directly specifies the model when submitting a task (such as --gpu-type=a100); Implicit matching: The system automatically recommends based on task metadata.

[0084] 2. Determine the number of GPUs

[0085] Computing based on task parallelism: For data parallel requirements, the number of GPUs required = ceil (total data volume / maximum batch size of a single card); for model parallel requirements, the number of GPUs required = number of model layers / number of layers carried by each GPU.

[0086] 3. Quantifying GPU Resource Requirements

[0087] GPU resource requirement = GPU model * quantity * 10,000, where 10,000 is the expansion factor.

[0088] In this embodiment, by closely combining hardware specifications and task characteristics, accurate GPU resource demand assessment and allocation can be achieved.

[0089] Step S103: Obtain the available resource amount of each node in the Kubernetes cluster, and sort the nodes in descending order of available resource amount to obtain a third queue.

[0090] In this embodiment, after reordering, the tasks in the second queue are dequeued and submitted to the scheduler, and the task dispatcher in the scheduler receives the tasks. The node load module of the scheduler sorts the nodes in descending order of available resources to obtain the third queue.

[0091] Optionally, in this embodiment, the native kubectl command can be used to view the total number of GPUs of all nodes and the number of allocated GPUs. The available resources of each node are the difference between the total number of GPUs of the node and the number of allocated GPUs.

[0092] Step S104: Each of the tasks is matched with each of the nodes in the third queue in the order of the second queue for multiple rounds. Each of the tasks in the first interval is matched from the first node in the third queue backward, and each of the tasks in the second interval is matched from the last node in the third queue forward, until all the tasks are matched or the resources are exhausted.

[0093] In this embodiment, the first queue dynamically adjusts the queue order to form a second queue based on the large task requirements and the ability to meet resource specification requirements. The tasks in the second queue are matched with nodes with large resource quantities first, and are matched from large to small. After a successful match, they are matched again from large to small until all tasks are matched or the resources are exhausted.

[0094] Specifically:

[0095] 1. First round of matching:

[0096] Sort all pending tasks by GPU requirements from largest to smallest (e.g. 8-GPU task → 2-GPU task → 1-GPU task).

[0097] Sort the available nodes by remaining GPU resources from largest to smallest (for example, Node1 has 8 GPUs remaining → Node2 has 8 GPUs remaining → Node3 has 4 GPUs remaining).

[0098] Try to match the largest tasks to the largest available nodes:

[0099] If the match is successful (for example, an 8-GPU task is assigned to Node1), the task is removed from the queue and the node resources are updated (Node1 has 0 GPUs remaining).

[0100] If the matching fails (for example, the current largest node has only 4 GPUs, but the task requires 8 GPUs), the task is skipped.

[0101] 2. Second round of matching:

[0102] After the first round of matching is complete, recheck the remaining resources and task queues:

[0103] The remaining node resources may have changed (for example, Node1 is occupied, but Node2 still has 8 GPUs left).

[0104] There may still be unassigned tasks in the task queue (for example, tasks that were skipped due to insufficient resources).

[0105] Repeat the first round of matching logic: sort the remaining tasks from largest to smallest again, and sort the remaining nodes from largest to smallest.

[0106] 3. Continue trying to match until all tasks are assigned or resources are exhausted.

[0107] Specific scenario examples:

[0108] Example 1

[0109] Assume the initial state: task queue (sorted by GPU demand): task B (8 GPUs), task C (2 GPUs), task A (1 GPU).

[0110] Available nodes: Node1 (8 GPUs), Node2 (8 GPUs), Node3 (4 GPUs).

[0111] First round of matching:

[0112] Task B (8 GPUs) attempts to match Node 1 (8 GPUs) → Successfully, Node 1 is fully occupied.

[0113] Remaining tasks: Task C (2 GPUs), Task A (1 GPU)

[0114] Remaining nodes: Node2 (8 GPUs), Node3 (4 GPUs)

[0115] Second round of matching (again from largest to smallest):

[0116] Reorder the remaining tasks: Task C (2 GPUs) → Task A (1 GPU)

[0117] Reorder the remaining nodes: Node2 (8 GPUs) → Node3 (4 GPUs)

[0118] Task C (2 GPUs) attempts to match Node3 (4 GPUs) → Allocation succeeds, and Node3 has 2 GPUs remaining.

[0119] Remaining tasks: Task A (1 GPU)

[0120] Remaining node resources: Node2 (8 GPUs)

[0121] Finally, Task A is assigned to the remaining 8 GPUs of Node2 (if sharing is allowed), or waits for new resources to be released.

[0122] In this embodiment, this design enables tasks that require more GPUs (such as 8-GPU tasks) to be given priority, reducing the waiting time of large tasks and avoiding small tasks occupying large resources (such as using two 8-GPU nodes to run small tasks), thereby improving overall resource utilization.

[0123] Optionally, in this embodiment, the second queue is divided into a first interval segment and a second interval segment according to resource requirements, and the resource requirement of each task in the first interval segment is greater than the resource requirement of each task in the second interval segment; each task in the first interval segment is matched backward from the first node in the third queue, that is, the node with the largest available resources, and each task in the second interval segment is matched forward from the last node in the third queue, that is, the node with the smallest available resources.

[0124] Exemplarily, the first interval segment is [5-8] GPU requirements, and the second interval segment is [1-4] GPU requirements.

[0125] For example, in Example 1, task B (8 GPUs) is matched backward from Node 1 (8 GPUs), and task C (2 GPUs) and task A (1 GPU) are matched forward from Node 3 (4 GPUs).

[0126] In this embodiment, the design is used to place small tasks on small nodes to reserve large nodes for large tasks, so as to further avoid small tasks occupying large resources, prevent resource fragmentation, and improve resource utilization.

[0127] Optionally, in this embodiment, when a new task appears after the first round of matching, the system dynamically handles it through the following mechanism:

[0128] 1. New tasks trigger instant queue updates

[0129] Insertion into the queue: New tasks are inserted into the task queue according to their priority (e.g., online tasks have high priority, offline tasks have low priority).

[0130] Example:

[0131] If the new task is online task D (4 GPUs, high priority) → insert it into the queue head

[0132] If the new task is offline task E (3GPU, low priority) → insert it to the end of the queue

[0133] 2. Dynamically interrupt the current scheduling cycle

[0134] Strategy selection:

[0135] The processing method for high-priority tasks is: wait until the current round of matching is completed before triggering the next round of matching;

[0136] The low-priority task is handled by waiting for the current cycle to complete and then restarting the matching process.

[0137] Operation logic:

[0138] If the new task is a high-priority task (such as an online service), the system will suspend the current task allocation (tasks that have not been submitted for scheduling will remain in the same state), add the new task to the queue and reorder it (Task D → Task B → Task C → Task A → Task E), and immediately start a new round of "matching from large to small".

[0139] If the new task is a low-priority task (such as offline computing), the system will add it to the end of the queue, continue to complete the resource allocation of the current scheduling cycle, and process the new task in the next scheduling cycle.

[0140] In this embodiment, the resource availability of each node in the Kubernetes cluster is updated at any time as each node is occupied and released, and the nodes are sorted in descending order according to the resource availability.

[0141] Step S105: sending each of the tasks to each of the nodes for processing according to the matching results.

[0142] From the above description, it can be seen that the large-scale priority scheduling method for GPU resources provided by the embodiment of the present application dynamically adjusts the queue order based on the large task requirements and the ability to meet the resource specification requirements, and gives priority to matching according to the large resource amount, from large to small. After the match is successful, it is matched again from large to small, giving priority to allowing large tasks to completely occupy large nodes, and small tasks are concentrated in small nodes where resources are matched. This can avoid the problem of small tasks occupying large resources and large tasks running across nodes, prevent resource fragmentation, reduce the communication cost of tasks across nodes and resource management overhead, and improve resource utilization.

[0143] In order to improve the resource utilization of cluster nodes and avoid GPU resource fragmentation, the present application provides an embodiment of a large-scale priority scheduling device for GPU resources for implementing all or part of the content of the large-scale priority scheduling method for GPU resources, see Figure 3 The large-scale priority scheduling device for GPU resources specifically includes the following contents:

[0144] The task receiving module 10 is used to receive tasks submitted by users and arrange the tasks into a first queue according to priority;

[0145] The task adjustment module 20 is configured to obtain resource requirements of each task in the first queue, and adjust the order of the first queue from largest to smallest based on the resource requirements of the tasks to obtain a second queue; divide the second queue into a first interval and a second interval according to the resource requirements, wherein the resource requirements of each task in the first interval are greater than the resource requirements of each task in the second interval;

[0146] The node adjustment module 30 is used to obtain the available resources of each node in the Kubernetes cluster and sort the nodes in descending order of available resources to obtain a third queue;

[0147] a matching module 40 configured to perform multiple rounds of matching of each task with each node in the third queue in the order of the second queue, wherein each task in the first interval is matched from the first node in the third queue backward, and each task in the second interval is matched from the last node in the third queue forward, until all tasks are matched or resources are exhausted;

[0148] The scheduling module 50 is used to send each of the tasks to each of the nodes for processing according to the matching results.

[0149] From the above description, it can be seen that the large-scale priority scheduling device for GPU resources provided by the embodiment of the present application dynamically adjusts the queue order based on the large task requirements and the ability to meet the resource specification requirements, and gives priority to matching according to the large resource amount, from large to small. After the match is successful, it is matched again from large to small, giving priority to allowing large tasks to completely occupy large nodes, and small tasks are concentrated in small nodes where resources are matched. This can avoid the problem of small tasks occupying large resources and large tasks running across nodes, prevent resource fragmentation, reduce the communication cost of tasks across nodes and resource management overhead, and improve resource utilization.

[0150] From a hardware perspective, in order to improve cluster node resource utilization and avoid GPU resource fragmentation, this application provides an embodiment of an electronic device for implementing all or part of the large-scale priority scheduling method for GPU resources. The electronic device specifically includes the following:

[0151] A processor, a memory, a communications interface, and a bus; wherein the processor, memory, and communications interface communicate with each other via the bus; the communications interface is used to transmit information between a large-scale priority scheduling device for GPU resources and related devices such as core business systems, user terminals, and related databases; the logic controller can be a desktop computer, a tablet computer, a mobile terminal, etc., but this embodiment is not limited thereto. In this embodiment, the logic controller can be implemented with reference to the embodiment of the large-scale priority scheduling method for GPU resources and the embodiment of the large-scale priority scheduling device for GPU resources in the embodiment, and their contents are incorporated herein and repeated parts are not repeated.

[0152] It is understandable that the user terminal may include a smart phone, a tablet electronic device, a network set-top box, a portable computer, a desktop computer, a personal digital assistant (PDA), a vehicle-mounted device, a smart wearable device, etc. Among them, the smart wearable device may include smart glasses, a smart watch, a smart bracelet, etc.

[0153] In practical applications, portions of the large-scale priority scheduling method for GPU resources can be executed on the electronic device side as described above, or all operations can be performed on the client device. The specific selection can be based on the processing capabilities of the client device and the limitations of the user's usage scenario. This application does not impose any restrictions on this. If all operations are performed on the client device, the client device may also include a processor.

[0154] The client device may include a communication module (i.e., a communication unit) that can establish a communication connection with a remote server to implement data transmission with the server. The server may include a server on the task scheduling center side, and in other implementation scenarios, may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, a server cluster consisting of multiple servers, or a server structure of a distributed device.

[0155] Figure 4 Schematic block diagram of the system structure of the electronic device 9600 according to an embodiment of the present application. Figure 4 As shown, the electronic device 9600 may include a central processing unit 9100 and a memory 9140; the memory 9140 is coupled to the central processing unit 9100. It is worth noting that the Figure 4 is exemplary; other types of structures may also be used to supplement or replace this structure to implement telecommunication functions or other functions.

[0156] In one embodiment, the large-scale priority scheduling method for GPU resources can be integrated into the central processing unit 9100. The central processing unit 9100 can be configured to perform the following control:

[0157] Step S101: receiving tasks submitted by users and arranging the tasks into a first queue according to priority;

[0158] Step S102: Obtaining resource requirements of each task in the first queue, and adjusting the order of the first queue from largest to smallest based on the resource requirements of the tasks to obtain a second queue; dividing the second queue into a first interval and a second interval according to the resource requirements, wherein the resource requirements of each task in the first interval are greater than the resource requirements of each task in the second interval;

[0159] Step S103: Obtain the available resource amount of each node in the Kubernetes cluster, and sort the nodes in descending order of available resource amount to obtain a third queue;

[0160] Step S104: each task is matched with each node in the third queue in the order of the second queue for multiple rounds. Each task in the first interval is matched from the first node in the third queue backwards, and each task in the second interval is matched from the last node in the third queue forwards, until all tasks are matched or resources are exhausted.

[0161] Step S105: sending each of the tasks to each of the nodes for processing according to the matching results.

[0162] From the above description, it can be seen that the electronic device provided in the embodiment of the present application dynamically adjusts the queue order based on the large task requirements and the ability to meet resource specification requirements, and gives priority to matching tasks with large resource amounts, from large to small. After a successful match, it matches again from large to small, giving priority to allowing large tasks to completely occupy large nodes, and small tasks to be concentrated in small nodes where resources are matched. This can avoid the problem of small tasks occupying large resources and large tasks running across nodes, prevent resource fragmentation, reduce the communication cost of tasks across nodes and resource management overhead, and improve resource utilization.

[0163] In another embodiment, the large-scale priority scheduling device for GPU resources can be configured separately from the central processing unit 9100. For example, the large-scale priority scheduling device for GPU resources can be configured as a chip connected to the central processing unit 9100, and the function of the large-scale priority scheduling method for GPU resources can be realized through the control of the central processing unit.

[0164] like Figure 4As shown, the electronic device 9600 may further include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is worth noting that the electronic device 9600 does not necessarily have to include Figure 4 In addition, the electronic device 9600 may also include all components shown in Figure 4 For components not shown, reference may be made to the prior art.

[0165] like Figure 4 As shown, the central processing unit 9100 is sometimes also referred to as a controller or operation control, and may include a microprocessor or other processor device and / or logic device. The central processing unit 9100 receives input and controls the operation of various components of the electronic device 9600.

[0166] Memory 9140 can be, for example, one or more of a cache, flash memory, hard drive, removable media, volatile memory, non-volatile memory, or other suitable devices. It can store the aforementioned failure-related information and also store programs that execute the relevant information. The CPU 9100 can execute the programs stored in memory 9140 to implement information storage or processing.

[0167] The input unit 9120 provides input to the central processing unit 9100. The input unit 9120 may be, for example, a keypad or touch input device. The power supply 9170 is used to provide power to the electronic device 9600. The display 9160 is used to display objects such as images and text. The display may be, for example, an LCD display, but is not limited thereto.

[0168] The memory 9140 may be a solid-state memory, such as a read-only memory (ROM), a random access memory (RAM), or a SIM card. Alternatively, it may be a memory that retains information even when power is off, can be selectively erased, and is provided with more data. Examples of such memory are sometimes referred to as EPROMs. The memory 9140 may also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application / function storage unit 9142 for storing application programs and function programs or processes for executing the operation of the electronic device 9600 by the central processing unit 9100.

[0169] The memory 9140 may also include a data storage unit 9143 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 9144 of the memory 9140 may include various driver programs for communication functions of the electronic device and / or for executing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0170] The communication module 9110 is a transmitter / receiver that transmits and receives signals via the antenna 9111. The communication module 9110 (transmitter / receiver) is coupled to the central processor 9100 to provide input signals and receive output signals, which may be the same as the case of a conventional mobile communication terminal.

[0171] Based on different communication technologies, multiple communication modules 9110 can be provided in the same electronic device, such as a cellular network module, a Bluetooth module, and / or a wireless local area network module. The communication module 9110 (transmitter / receiver) is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide audio output via the speaker 9131 and receive audio input from the microphone 9132, thereby implementing typical telecommunication functions. The audio processor 9130 may include any suitable buffer, decoder, amplifier, etc. Furthermore, the audio processor 9130 is also coupled to the central processing unit 9100, enabling local recording via the microphone 9132 and playback of stored audio via the speaker 9131.

[0172] Embodiments of the present application also provide a computer-readable storage medium capable of implementing all steps of the large-size priority scheduling method for GPU resources in the above-mentioned embodiment, where the execution subject is a server or a client. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, all steps of the large-size priority scheduling method for GPU resources in the above-mentioned embodiment are implemented. For example, when the processor executes the computer program, the following steps are implemented:

[0173] Step S101: receiving tasks submitted by users and arranging the tasks into a first queue according to priority;

[0174] Step S102: Obtaining resource requirements of each task in the first queue, and adjusting the order of the first queue from largest to smallest based on the resource requirements of the tasks to obtain a second queue; dividing the second queue into a first interval and a second interval according to the resource requirements, wherein the resource requirements of each task in the first interval are greater than the resource requirements of each task in the second interval;

[0175] Step S103: Obtain the available resource amount of each node in the Kubernetes cluster, and sort the nodes in descending order of available resource amount to obtain a third queue;

[0176] Step S104: each task is matched with each node in the third queue in the order of the second queue for multiple rounds. Each task in the first interval is matched from the first node in the third queue backwards, and each task in the second interval is matched from the last node in the third queue forwards, until all tasks are matched or resources are exhausted.

[0177] Step S105: sending each of the tasks to each of the nodes for processing according to the matching results.

[0178] From the above description, it can be seen that the computer-readable storage medium provided in the embodiment of the present application dynamically adjusts the queue order based on the large task requirements and the ability to meet resource specification requirements, and gives priority to matching tasks with large resource quantities, from large to small. After a successful match, it matches again from large to small, giving priority to allowing large tasks to completely occupy large nodes, and small tasks to be concentrated in small nodes where resources are matched. This can avoid the problem of small tasks occupying large resources and large tasks running across nodes, prevent resource fragmentation, reduce the communication cost of tasks across nodes and resource management overhead, and improve resource utilization.

[0179] Embodiments of the present application also provide a computer program product capable of implementing all steps of the large-size priority scheduling method for GPU resources in the above-mentioned embodiment, where the execution subject is a server or a client. When the computer program / instructions are executed by a processor, the steps of the large-size priority scheduling method for GPU resources are implemented. For example, the computer program / instructions implement the following steps:

[0180] Step S101: receiving tasks submitted by users and arranging the tasks into a first queue according to priority;

[0181] Step S102: Obtaining resource requirements of each task in the first queue, and adjusting the order of the first queue from largest to smallest based on the resource requirements of the tasks to obtain a second queue; dividing the second queue into a first interval and a second interval according to the resource requirements, wherein the resource requirements of each task in the first interval are greater than the resource requirements of each task in the second interval;

[0182] Step S103: Obtain the available resource amount of each node in the Kubernetes cluster, and sort the nodes in descending order of available resource amount to obtain a third queue;

[0183] Step S104: each task is matched with each node in the third queue in the order of the second queue for multiple rounds. Each task in the first interval is matched from the first node in the third queue backwards, and each task in the second interval is matched from the last node in the third queue forwards, until all tasks are matched or resources are exhausted.

[0184] Step S105: sending each of the tasks to each of the nodes for processing according to the matching results.

[0185] From the above description, it can be seen that the computer program product provided in the embodiment of the present application dynamically adjusts the queue order based on the large task requirements and the ability to meet resource specification requirements, prioritizes matching according to the large amount of resources, and then matches from large to small. After a successful match, it matches again from large to small, giving priority to allowing large tasks to completely occupy large nodes, and small tasks are concentrated in small nodes where resources are matched. This can avoid the problem of small tasks occupying large resources and large tasks running across nodes, prevent resource fragmentation, reduce the communication cost of tasks across nodes and resource management overhead, and improve resource utilization.

[0186] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, apparatus, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0187] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (apparatus), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as a combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0188] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0189] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0190] Specific embodiments are used in the present invention to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the contents of this specification should not be understood as limiting the present invention.

Claims

1. A large-scale priority scheduling method for GPU resources, characterized in that: The method comprises: Receive tasks submitted by users and arrange the tasks into a first queue according to priority; Obtaining resource requirements of each of the tasks in the first queue, and adjusting the order of the first queue from largest to smallest based on the resource requirements of the tasks to obtain a second queue; dividing the second queue into a first interval and a second interval according to the resource requirements, wherein the resource requirements of each of the tasks in the first interval are greater than the resource requirements of each of the tasks in the second interval; Obtain the available resources of each node in the Kubernetes cluster, and sort the nodes in descending order of available resources to obtain a third queue; Each task is matched with each node in the third queue in multiple rounds according to the order of the second queue. Each task in the first interval is matched from the first node in the third queue backward, and each task in the second interval is matched from the last node in the third queue forward, until all tasks are matched or resources are exhausted. According to the matching results, each of the tasks is sent to each of the nodes for processing.

2. The large-scale priority scheduling method for GPU resources according to claim 1, characterized in that: The step of performing multiple rounds of matching of the tasks with the nodes in the third queue in the order of the second queue includes: If the match is successful, the task is removed from the second queue and the available resources of the node are updated; If the match fails, the task is skipped.

3. The large-scale priority scheduling method for GPU resources according to claim 1, characterized in that: The step of performing multiple rounds of matching of each of the tasks with each of the nodes in the third queue in the order of the second queue further includes: Receive a new task submitted by a user, and insert the new task into the first queue according to its priority, to obtain a new first queue; After the current round of matching is completed, the next round of matching is triggered, or after the current scheduling cycle is completed, the matching process is restarted; Obtaining resource requirements of each of the tasks in the new first queue, and adjusting the order of the new first queue from largest to smallest based on the resource requirements of the tasks to obtain a new second queue; Each of the tasks is matched with each of the nodes in the third queue in the order of the new second queue for multiple rounds until all the tasks are matched or the resources are exhausted.

4. The large-scale priority scheduling method for GPU resources according to claim 1, characterized in that: The step of performing multiple rounds of matching of each of the tasks with each of the nodes in the third queue in the order of the second queue further includes: Re-obtain the available resources of each node in the Kubernetes cluster, and sort the nodes in descending order of available resources to obtain a new third queue; Each of the tasks is matched with each of the nodes in the new third queue in the order of the second queue for multiple rounds until all the tasks are matched or the resources are exhausted.

5. The large-scale priority scheduling method for GPU resources according to claim 1, characterized in that: The step of arranging the tasks into a first queue according to priority comprises: Determining the type of the task submitted by the user and determining a priority based on the task type; the task types include tasks requiring immediate response, tasks that allow for delay, and tasks requiring a moderate response time; Extracting time-related attributes of the tasks, and further determining the priorities of the tasks of each type based on the time-related attributes; the time-related attributes include submission time, estimated execution time, deadline, and service level agreement.

6. The large-scale priority scheduling method for GPU resources according to claim 1, characterized in that: The step of arranging the tasks into a first queue according to priority also includes: Performing multi-dimensional weight assignment on the task type and the time-related attributes; Setting the conditions for recalculating the priority; the conditions include: fixed period, new task arrival; Set preemption rules: new tasks have higher priority than running tasks, and preempted tasks support state saving; Set up a time decay mechanism: gradually increase the priority of tasks that have been waiting for a long time.

7. The large-scale priority scheduling method for GPU resources according to claim 1, characterized in that: The step of obtaining the resource requirements of each task in the first queue includes: Specifying a GPU model according to the characteristics of the task submitted by the user; Determining the number of GPUs based on the parallelism of the task; The resource requirement of the task is calculated based on the GPU model and the number of GPUs.

8. A large-scale priority scheduling device for GPU resources, characterized in that: The device comprises: A task receiving module is used to receive tasks submitted by users and arrange the tasks into a first queue according to priority; a task adjustment module, configured to obtain resource requirements of each of the tasks in the first queue, and adjust the order of the first queue from largest to smallest based on the resource requirements of the tasks to obtain a second queue; divide the second queue into a first interval and a second interval according to the resource requirements, wherein the resource requirements of each of the tasks in the first interval are greater than the resource requirements of each of the tasks in the second interval; A node adjustment module is used to obtain the available resources of each node in the Kubernetes cluster and sort the nodes in descending order of available resources to obtain a third queue; a matching module configured to perform multiple rounds of matching of each task with each node in the third queue in the order of the second queue, wherein each task in the first interval is matched from the first node in the third queue backward, and each task in the second interval is matched from the last node in the third queue forward, until all tasks are matched or resources are exhausted; The scheduling module is used to send each of the tasks to each of the nodes for processing according to the matching results.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the large-scale priority scheduling method for GPU resources described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the large-scale priority scheduling method for GPU resources described in any one of claims 1 to 7 are implemented.

Images