A method and system for allocating computing resources through heterogeneous GPU hybrid scheduling

Abstract

This invention discloses a method and system for heterogeneous GPU hybrid scheduling of computing resources, belonging to the field of computer equipment technology. The key technical points include: identifying the allocation type corresponding to the task to be allocated, including single task allocation, group task allocation, and urgent task allocation; if the allocation type is group task allocation, obtaining the priority of each task in the group to be allocated; allocating the first task to a GPU node according to the priority, so that the GPU node generates an affinity label between the first task and the unallocated tasks, the affinity label responding to the identity label of the unallocated tasks; and allocating the unallocated tasks to GPU nodes according to the affinity label. This invention obtains the affinity label through a response mechanism and performs task allocation based on the affinity label, which reduces allocation time and improves scheduling efficiency compared to traversing a lookup table to obtain the affinity relationship of unallocated tasks and then performing allocation.

Description

Technical Field

[0001] This invention relates to the field of computer equipment technology, and more specifically to a method and system for allocating computing resources through heterogeneous GPU hybrid scheduling. Background Technology

[0002] With the development of artificial intelligence, big data and other fields, heterogeneous GPUs are widely used to meet complex computing needs. Traditional resource scheduling methods often fail to accurately adapt to task and resource characteristics when dealing with heterogeneous resources, resulting in low resource utilization and poor task execution efficiency. Furthermore, current technologies mostly focus on single resource types or simple task scheduling, making it difficult to cope with complex and diverse task loads and heterogeneous resource combinations. Therefore, existing technologies have shortcomings. Summary of the Invention

[0003] To address the shortcomings of existing technologies, the present invention aims to provide a method and system for allocating computing resources through heterogeneous GPU hybrid scheduling. By setting priorities, affinity tags, and response mechanisms, it enables the scheduling of computing resources under different allocation types.

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] This invention provides a method for allocating computing resources in a heterogeneous GPU hybrid scheduling system. The computing resource allocation method is executed by a scheduler, which is communicatively connected to multiple GPU nodes, and the multiple GPU nodes are communicatively connected to each other. The computing resource allocation method includes:

[0006] Identify the assignment type corresponding to the task to be assigned, including single task assignment, group task assignment, and emergency task assignment;

[0007] If the allocation type is group task allocation, obtain the priority of each task in the task group to be allocated;

[0008] The first task is assigned to a GPU node according to the priority, so that the GPU node generates an affinity label between the first task and the unassigned task. The first task is the highest priority task in the current task group to be assigned, and the affinity label responds to the identity label of the unassigned task.

[0009] The unassigned tasks are assigned to GPU nodes based on the affinity tag.

[0010] As a further improvement of the present invention, the step of allocating the unassigned task to the GPU node according to the affinity tag includes:

[0011] For the unassigned task, perform a first iteration operation, which includes: obtaining a second task, broadcasting the identity tag of the second task to all GPU nodes so that GPU nodes with an affinity tag related to the second task respond to the identity tag of the second task and send the affinity tag, and assigning the second task to a GPU node according to the affinity tag; the second task is the highest priority task among the currently unassigned tasks.

[0012] As a further improvement of the present invention, the step of assigning the second task to a GPU node based on the affinity tag includes:

[0013] If a hard affinity label exists among the affinity labels, the second task is assigned to the GPU node corresponding to the hard affinity label;

[0014] If there is no hard affinity label among the affinity labels, but there is a hard anti-affinity label, the second task is assigned to a GPU node other than the GPU node corresponding to the hard anti-affinity label.

[0015] As a further improvement of the present invention, after assigning the second task to a GPU node according to the affinity tag, the computing resource allocation method further includes:

[0016] The second task is defined as the third task, so that the GPU node generates an affinity label between the third task and the currently unassigned task.

[0017] As a further improvement of the present invention, while broadcasting the identity label of the second task to all GPU nodes, the computing resource allocation method further includes:

[0018] The first data in the second task is transmitted to GPU nodes within a preset distance, wherein the time for transmitting the first data in the second task to GPU nodes within a preset distance is equal to the time for broadcasting the identity tag of the second task to all GPU nodes, and the first data is the data in the second task that is located at a preset position.

[0019] As a further improvement of the present invention, the step of assigning the second task to a GPU node based on the affinity tag includes:

[0020] Based on the affinity tag, the second data in the second task is transmitted to the GPU node;

[0021] Send a transmission command to the GPU node that is closest to the GPU node to which the second task is assigned among the GPU nodes within the preset distance, so that the GPU node sends the first data to the GPU node to which the second task is assigned;

[0022] A deletion command is sent to GPU nodes within the preset distance to cause the GPU nodes within the preset distance to delete the first data.

[0023] As a further improvement of the present invention, the computing resource allocation method further includes:

[0024] If the allocation type is a single task allocation, obtain the priority and computing power configuration requirements of the task to be allocated;

[0025] According to the priority, the task to be assigned is placed into the corresponding priority queue;

[0026] A second iteration operation is performed on the priority queue. The second iteration operation includes: retrieving a fourth task from the priority queue; determining whether there is a GPU node that meets the computing power configuration requirements of the fourth task and the resource usage of each GPU node based on the computing power configuration requirements of the fourth task and the resource usage of each GPU node; if not, allocating a GPU node to the fourth task according to the retry mechanism and the preemption mechanism, until all tasks in the priority queue are allocated, wherein the fourth task is the task located at the head of the priority queue.

[0027] As a further improvement of the present invention, the allocation of GPU nodes for the fourth task based on the retry mechanism and the preemption mechanism includes:

[0028] The GPU node is allocated to the fourth task according to the preset retry interval. If the fourth task is still not allocated a GPU node when the number of retries reaches the preset number of retries, the retries are stopped.

[0029] Obtain the execution progress and computing power configuration requirements of the tasks to be preempted; determine the first target task based on the execution progress and computing power configuration requirements; the tasks to be preempted are determined based on the priority and computing power configuration requirements of the currently executing tasks.

[0030] Pause the first target task and assign the fourth task to the GPU node corresponding to the first target task.

[0031] As a further improvement of the present invention, the computing resource allocation method further includes:

[0032] If the allocation type is emergency task allocation, obtain the priority of the task to be allocated;

[0033] Determine the second target task from the currently executing tasks based on the priority;

[0034] The dependencies of the second target task are determined based on a preset directed acyclic graph. A GPU node is selected from the GPU nodes corresponding to the second target task, and the task to be assigned is assigned to the GPU node.

[0035] This invention provides a computing resource allocation system for heterogeneous GPU hybrid scheduling. The computing resource allocation system includes a scheduler and multiple GPU nodes. The scheduler is communicatively connected to the multiple GPU nodes, and the multiple GPU nodes are communicatively connected to each other.

[0036] The scheduler is used to identify the allocation type corresponding to the task to be assigned, which includes single task allocation, group task allocation, and urgent task allocation; if the allocation type is group task allocation, the priority of each task in the task group to be assigned is obtained; the first task is assigned to a GPU node according to the priority; and unassigned tasks are assigned to GPU nodes according to the affinity tag.

[0037] The GPU node is used to generate affinity tags for the first task and unassigned tasks. The first task is the highest priority task in the current task group to be assigned. The affinity tag is in response to the identity tag of the unassigned task.

[0038] This invention generates affinity tags when a task arrives at a GPU node, then obtains the affinity tags through a response mechanism, and finally allocates tasks based on the affinity tags. Compared to traversing a lookup table to obtain the affinity relationships of unassigned tasks and then allocating them, this invention reduces allocation time. Furthermore, this invention transmits some data to the GPU node while broadcasting the task's identity tag, reducing the amount and time of subsequent data transmission and further improving scheduling efficiency. Attached Figure Description

[0039] Figure 1 This is a flowchart of the method steps of the present invention;

[0040] Figure 2 A schematic diagram of the response mechanism;

[0041] Figure 3 This is a schematic diagram of the data transmission process;

[0042] Figure 4 A flowchart of the steps for assigning a single task. Detailed Implementation

[0043] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations thereof.

[0044] The term "and / or" in the following text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0045] like Figure 1 As shown in the figure, this application provides a method for allocating computing resources through heterogeneous GPU hybrid scheduling, including:

[0046] Identify the assignment type of the task to be assigned, which includes single task assignment, group task assignment, and emergency task assignment.

[0047] If the assignment type is group task assignment, obtain the priority of each task in the task group to be assigned;

[0048] The first task is assigned to a GPU node based on priority, so that the GPU node generates an affinity label between the first task and the unassigned tasks. The first task is the highest priority task in the current task group to be assigned, and the affinity label responds to the identity label of the unassigned task.

[0049] Unassigned tasks are assigned to GPU nodes based on affinity tags.

[0050] Heterogeneous GPUs refer to GPUs of different types, architectures, or performance characteristics, meaning that GPU nodes in the current scenario are suitable for different tasks or can meet different computing power requirements. The aforementioned computing resource allocation method is executed by a scheduler, which can be, for example, Kubernetes. The scheduler communicates with multiple GPU nodes, and these GPU nodes also communicate with each other. The allocation type corresponding to the task to be assigned is the allocation type required for that task. Different allocation types are needed for different tasks. Specifically, the group task allocation type refers to Gang-scheduling based on PodGroups (container groups). A PodGroup includes multiple Pods. The characteristic of this allocation type is that it needs to allocate sufficient computing resources (i.e., GPU nodes) to the entire PodGroup at once, avoiding situations where some Pods are allocated to GPU nodes first, while other Pods cannot be allocated due to insufficient computing resources, leading to task failure or inefficiency. In other words, group task allocation only occurs when the remaining computing resources can meet the computing power requirements of the entire PodGroup.

[0051] Specifically, affinity tags can be divided into hard affinity tags and hard anti-affinity tags. Hard affinity means that a Pod must run on the same node as other specific Pods. For example, if there are two Pods, one a database service and the other an application service, and they have frequent data interactions, hard affinity can be set to allow them to run on the same node to reduce network latency. Conversely, hard anti-affinity means that a Pod is prohibited from running on the same node as other specific Pods. For example, to improve the reliability and fault tolerance of an application, it is not desirable for multiple replicas of the same service to run on the same node, so as to prevent the service from becoming completely unavailable due to node failure. In this case, hard anti-affinity can be set. Furthermore, not all Pods need to have affinity tags set. Identity tags are used to identify and describe the characteristics of Pods so that GUP nodes can recognize the tasks to be assigned next.

[0052] In this embodiment, affinity tags are generated when a task arrives at the GPU node. Then, the affinity tags are obtained through a response mechanism, and finally, the task is assigned based on the affinity tags. Compared with traversing the lookup table to obtain the affinity relationships of unassigned tasks and then assigning them, the assignment time is reduced.

[0053] Furthermore, such as Figure 2 As shown, this embodiment provides a method for assigning unassigned tasks to GPU nodes based on affinity tags, including:

[0054] For unassigned tasks, perform a first iteration operation, which includes: obtaining a second task, broadcasting the identity tag of the second task to all GPU nodes so that GPU nodes with affinity tags related to the second task respond to the identity tag of the second task and send affinity tags, and assigning the second task to a GPU node according to the affinity tags; the second task is the highest priority task among the currently unassigned tasks.

[0055] Here, all GPU nodes refer to the computing resources currently available for this PodGroup, i.e., the currently remaining computing resources; in the first iteration, the currently unassigned tasks are all the tasks remaining after removing the first task from the task group to be assigned.

[0056] Specifically, if no affinity tag is received from the GPU node, it means that there is no affinity tag related to the second task. In this case, a GPU node that can meet the computing power requirements of the second task is selected from these GPU nodes, and the second task is assigned to that node. If there is a hard affinity tag among the received affinity tags, the second task is assigned to the GPU node corresponding to the hard affinity tag. If there is no hard affinity tag, but there is a hard anti-affinity tag, the second task is assigned to any GPU node that can meet the computing power requirements of the second task, except for the GPU node corresponding to the hard anti-affinity tag.

[0057] Furthermore, after assigning the second task to a GPU node based on the affinity tag, the second task needs to be defined as the third task so that the GPU node can generate an affinity tag for the third task with the currently unassigned task.

[0058] For example, suppose there exists a PodGroup that needs to be used for task allocation. This PodGroup includes multiple Pods (Pod1, Pod2, Pod3, Pod4). In this case, the PodGroup is the task group to be allocated, and each Pod in the PodGroup is a task in the task group to be allocated. The remaining computing resources are GPU node 1, GPU node 2, GPU node 3, and GPU node 4. Next, the priority of each Pod is obtained. The priority of a Pod is determined according to its corresponding business type. For example, for a PodGroup containing multiple microservices, the Pods of the core business logic have a higher priority, while the Pods of auxiliary monitoring or log collection have a lower priority.

[0059] Next, the highest priority task in the current task group to be assigned is designated as the first task. Let's assume the first task is Pod1. Pod1 is then assigned to a GPU node that meets its computing power requirements. Let's assume Pod1 is assigned to GPU node 1. GPU node 1 then generates affinity tags between Pod1 and the unassigned tasks (Pod2, Pod3, Pod4). Assuming Pod1 and Pod2 have a hard anti-affinity, meaning they cannot be assigned to the same node, the identity tag for Pod2 is "app=pod2-app", where app represents the tag key, p... od2-app represents the value of the identity label. The affinity label generated by GPU node 1 is "anti-affinity-pod2-app=true". Here, anti-affinity indicates that this is an anti-affinity constraint, pod2-app is the value of the identity label, and true means that the constraint is enabled. This label means that scheduling Pods with the identity label app=pod2-app is prohibited. Therefore, when the scheduler tries to deploy Pod2 to GPU node 1, it will skip the GPU node because of the affinity label of GPU node 1, ensuring that Pod1 and Pod2 are not on the same node.

[0060] Next, iterative operations are performed on the unassigned tasks. First, the highest priority task is extracted from the current unassigned tasks and designated as the second task, let's say Pod2. Then, the identity label of the second task is broadcast to all GPU nodes. When GPU node 1 receives this identity label, since the affinity label generated by GPU node 1 is related to the second task, GPU node 1 responds to the identity label and sends the affinity label to the scheduler. After receiving the affinity label, the scheduler assigns Pod2 to any GPU node other than GPU node 1 that can meet the computing power requirements of the second task, let's say Pod2 is assigned to GPU node 2. Then, Pod2 is transformed into the third task, and GPU node 2 generates affinity labels for the third task Pod2 with the currently unassigned tasks (Pod3 and Pod4). The above steps are then repeated until all Pods are assigned.

[0061] In this embodiment, after each task arrives at a GPU node, an affinity tag is generated between that task and other tasks awaiting assignment. A response mechanism is set up to send the affinity tags to the scheduler, which then assigns the task to the GPU node based on the affinity tags. In contrast, existing technologies typically store a lookup table of all affinity tags in the scheduler. When a task needs to be assigned, the lookup table must be searched for affinity relationships with the currently assigned task. For example, when assigning Pod2, the lookup table must be traversed to find that Pod2 has a hard anti-affinity with Pod1. Then, all GPU nodes are traversed to find that Pod1 is assigned to GPU node 1. Finally, Pod2 is assigned to any GPU node other than GPU node 1 that can meet the computing power requirements of the second task. Compared to existing methods, this embodiment can determine affinity relationships and perform task assignment without traversing the lookup table and all GPU nodes, saving resource scheduling time.

[0062] Furthermore, while broadcasting the identity label of the second task to all GPU nodes, the above-mentioned computing resource allocation method also includes:

[0063] The first data in the second task is transmitted to GPU nodes within a preset distance, wherein the time for transmitting the first data in the second task to GPU nodes within a preset distance is equal to the time for broadcasting the identity tag of the second task to all GPU nodes, and the first data is the data in the second task that is located at a preset position.

[0064] The preset distance and the size of the first data are determined based on the time it takes to broadcast the identity label of the second task to all GPU nodes.

[0065] Specifically, the preset position is the first position, also known as the starting position. If the second task is a text file, such as a configuration file, the data at the first position in the second task is the first few lines of the configuration file, and the specific number of lines is set according to the actual situation. If the second task is a binary file, the data at the first position is the file header data. For example, the file header data of an image file contains the image format, size, color information, etc. If the second task is a database service, such as a relational database, it can be sorted according to a certain index (such as a primary key index). The records at the beginning of the sequence are the data at the first position. For example, in a user table, sorting by user ID primary key, the user record with the smaller ID is the record at the beginning of the sequence. The specific number of records selected is set according to the actual situation. If the data in the second task exists in a specific memory data structure, such as an array storing integers, the elements with indexes of 0, 1, 2, etc., are the data at the first position. The specific number of indexes selected is set according to the actual situation.

[0066] Furthermore, this embodiment provides a step of assigning a second task to a GPU node based on an affinity tag, including:

[0067] Based on the affinity tag, the second data in the second task is transferred to the GPU node;

[0068] Send a transmission command to the GPU node that is closest to the GPU node to which the second task is assigned, within a preset distance, so that the GPU node sends the first data to the GPU node to which the second task is assigned;

[0069] Send a delete command to GPU nodes within a preset distance to cause the GPU nodes within the preset distance to delete the first data.

[0070] For example, continuing with the above example, such as Figure 3 As shown, the GPU nodes within the preset distance are GPU node 3 and GPU node 4. While broadcasting Pod2's identity tag to all GPU nodes, the first data needs to be transmitted to GPU nodes 3 and 4. Then, following the steps in the example above, GPU node 2 is selected for Pod2. Next, a transmission command is sent to the GPU node closest to GPU node 2 (GPU node 4) to instruct it to send the first data to GPU node 2. The second data from the second task is also transmitted to GPU node 2. The second data is the data obtained by removing the first data from the second task. The timing of GPU node 4 sending the first data to GPU node 2 coincides with the timing of the scheduler transmitting the second data from the second task to GPU node 2. After GPU node 2 receives the first data, it sends a deletion command to the GPU nodes within the preset distance to delete the first data.

[0071] In this embodiment, the first data of the second task is transmitted while the identity tag is broadcast, and the time required to transmit the first data is the same as the time required for broadcasting. Then, the remaining data is transmitted after the GPU node corresponding to the second task is determined. Compared with transmitting all data after the GPU node corresponding to the second task is determined, the transmission time is reduced and the allocation efficiency is improved.

[0072] Furthermore, such as Figure 4 As shown, the above-mentioned computing resource allocation method also includes:

[0073] If the allocation type is single task allocation, obtain the priority and computing power configuration requirements of the task to be allocated;

[0074] Based on priority, tasks to be assigned are placed into the corresponding priority queue;

[0075] The second iteration operation is performed on the priority queue. The second iteration operation includes: taking the fourth task out of the priority queue, determining whether there is a GPU node that meets the computing power configuration requirements of the fourth task and the resource usage of each GPU node based on the computing power configuration requirements of the fourth task and the resource usage of each GPU node. If not, a GPU node is allocated to the fourth task according to the retry mechanism and the preemption mechanism, until the allocation of all tasks in the priority queue is completed. The fourth task is the task located at the head of the priority queue.

[0076] The computing power configuration requirement refers to the computing power resources required to complete the task.

[0077] Specifically, before placing the task into the priority queue, it is first determined whether there is a GPU node that can meet the computing power configuration requirements of the task. If there is, the task to be assigned is directly assigned to that GPU node. If there are multiple GPU nodes, the task to be assigned is assigned to any one of these GPU nodes without needing to place it into the priority queue. If there is no GPU node, it is placed into the priority queue to wait.

[0078] The priority of tasks to be assigned is determined based on business importance, time sensitivity, and resource requirements. Tasks related to core business usually have higher priority. For example, in an e-commerce system, the task of processing order payments has a higher priority than the task of sending marketing emails. Tasks with strict time requirements have higher priority. For example, the task of generating daily reports on a schedule must be completed at a specific time, so it will have a higher priority. Tasks with high and urgent demand for critical resources have higher priority.

[0079] Next, the tasks to be assigned are placed into the corresponding priority queue, and it is checked whether a parent task exists in the priority queue. If a parent task exists, its priority is checked, and the priority of the current task to be assigned is updated to the parent task's priority. The position of the task to be assigned is then updated according to the updated priority, and the updated position should be after the parent task. A parent task is a task that has a dependency on other tasks and logically executes before them. For example, the task "Generate Order Report" depends on the completion of the prerequisite tasks "Get Order Data" and "Analyze Order Data." Therefore, "Get Order Data" and "Analyze Order Data" are considered the parent tasks of the "Generate Order Report" task. Only after the parent tasks are completed can the subtask "Generate Order Report" be executed smoothly. To further improve efficiency, a parent task identifier (such as task ID) can be added to each task to be assigned. If this field has a value, it indicates that a parent task exists; if it is empty, it indicates that no parent task exists.

[0080] Then, an iterative operation is performed on each task in the priority queue, and a GPU node is allocated to each task.

[0081] Furthermore, this embodiment provides a step for allocating a GPU node to a fourth task based on a retry mechanism and a preemption mechanism, including:

[0082] The GPU node is allocated to the fourth task according to the preset retry interval. If the fourth task has not been allocated a GPU node when the preset retry count is reached, the retrying stops.

[0083] Obtain the execution progress and computing power configuration requirements of the tasks to be preempted. Based on the execution progress and computing power configuration requirements, determine the first target task. The tasks to be preempted are determined based on the priority and computing power configuration requirements of the currently executing tasks.

[0084] Pause the first target task and assign the fourth task to the GPU node corresponding to the first target task.

[0085] Specifically, if no GPU node exists that can meet the computing power configuration requirements of the fourth task, a retry mechanism is initiated. This means that after a preset retry interval, the system re-evaluates whether a GPU node capable of meeting the computing power configuration requirements of the fourth task exists. The preset retry interval is determined based on the current memory usage of the GPU node. For example, if the memory usage of a GPU node gradually decreases from 80% to 30%, it indicates that the GPU node's resources are about to be released, and the retry interval can be shortened to increase the chance of the fourth task being allocated. Conversely, the retry interval is increased to avoid resource waste caused by frequent retries. The 80% and 30% values ​​are merely examples, and this embodiment does not impose any limitations on the specific values.

[0086] If the fourth task has not been assigned a GPU node after the preset number of retries has been reached, and the fourth task has a higher priority, a preemption mechanism is executed. The number of retries is determined based on the priority and urgency of the fourth task; the higher the priority and urgency, the fewer the retries. Next, the priorities and computing power requirements of all currently executing tasks are iterated. Tasks with lower priorities than the fourth task but higher computing power requirements are grouped into task groups. It is determined whether the GPU node corresponding to each task in the task group can meet the computing power requirements of the fourth task after pausing the task. Tasks corresponding to GPU nodes that can meet these requirements are designated as preemptible tasks. The execution progress of each preemptible task is then obtained, and the task with the slowest execution progress is selected as the first target task. Finally, the first target task is paused, and the fourth task is assigned to the GPU node corresponding to the first target task. After the fourth task completes its execution, the GPU node continues to complete the first target task.

[0087] In this embodiment, when allocating GPU nodes to a task, the resource usage of each GPU node is first used to determine whether the computing power configuration requirements of the task can be met. At the same time, a retry mechanism is introduced when the requirements cannot be met, giving the task multiple opportunities to acquire resources, thereby further improving resource utilization. When the number of retries reaches the preset number of retries and a GPU node still cannot be allocated, a preemption mechanism is activated. The execution progress and computing power configuration requirements of the tasks to be preempted are obtained to determine the first target task, so as to ensure the smooth execution of high-priority tasks.

[0088] Furthermore, the above-mentioned computing resource allocation methods also include:

[0089] If the assignment type is emergency task assignment, obtain the priority of the task to be assigned;

[0090] Determine the second target task from the currently executing tasks based on priority;

[0091] The dependencies of the second target task are determined based on the preset directed acyclic graph. A GPU node is selected from the GPU nodes corresponding to the second target task, and the task to be assigned is assigned to the GPU node.

[0092] Specifically, if the task to be assigned is a task with extremely high commercial value or has a critical impact on the operation of the system, even if its original priority is not the highest, but it is determined according to specific rules that it needs to be executed immediately, then the assignment type corresponding to the task to be assigned is emergency task assignment.

[0093] Next, the priority of the task is obtained, and tasks with lower priority than the currently executing task are identified and designated as second target tasks. The dependencies of each second target task are then determined based on a pre-defined directed acyclic graph (DAG). The DAG stores these dependencies; nodes represent tasks, and directed edges represent dependencies. For example, the execution of task A depends on the data output of task B, and the execution of task B depends on the computation result of task C. This chain of relationships is then generated into a DAG and stored in the scheduler. The dependencies between the second target task and other tasks are then determined based on the DAG. If another higher-priority task depends on the second target task, the GPU node corresponding to that second target task is excluded. Finally, a GPU node is randomly selected from the remaining nodes corresponding to the tasks, the task currently being processed is paused, and the unassigned task is assigned to that GPU node. Once the unassigned task completes, the GPU node resumes its previous task.

[0094] This embodiment quickly locks low-priority tasks in the currently executing tasks based on the priority of the tasks to be assigned. It can quickly allocate GPU node resources for urgent tasks, realize a rapid response to urgent needs, and shorten task waiting time. Furthermore, this embodiment combines a preset directed acyclic graph to accurately analyze task dependencies and select GPU nodes based on this. This not only ensures the rationality of resource allocation and reduces interference with existing business processes, but also effectively avoids scheduling chaos caused by unclear dependencies.

[0095] Furthermore, embodiments of this application provide a computing resource allocation system for heterogeneous GPU hybrid scheduling. The computing resource allocation system includes a scheduler and multiple GPU nodes. The scheduler is communicatively connected to the multiple GPU nodes, and the multiple GPU nodes are communicatively connected to each other.

[0096] The scheduler identifies the assignment type of the task to be assigned, which includes single task assignment, group task assignment, and urgent task assignment. If the assignment type is group task assignment, the scheduler obtains the priority of each task in the group to be assigned. The scheduler assigns the first task to a GPU node according to the priority. The scheduler assigns unassigned tasks to GPU nodes according to the affinity tag.

[0097] GPU nodes are used to generate affinity tags for the first task and unassigned tasks. The first task is the highest priority task in the current task group to be assigned, and the affinity tag responds to the identity tag of the unassigned task.

[0098] This application provides a method and system for allocating computing resources in a heterogeneous GPU hybrid scheduling manner. When a task arrives at a GPU node, an affinity tag is generated. Then, the affinity tag is obtained through a response mechanism, and finally, the task is allocated based on the affinity tag. Compared with traversing a lookup table to obtain the affinity relationship of unassigned tasks and then allocating them, the allocation time is reduced. Furthermore, this embodiment transmits first data to nodes within a preset distance while broadcasting the identity tag, and controls the node to transmit the first data to the node to which the task is assigned while transmitting the second data, further saving the time required for the transmission process.

[0099] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0100] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0101] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0102] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A method for allocating computing resources through heterogeneous GPU hybrid scheduling, characterized in that, The computing resource allocation method is executed by a scheduler, which is communicatively connected to multiple GPU nodes, and the multiple GPU nodes are communicatively connected to each other. The computing resource allocation method includes: Identify the assignment type corresponding to the task to be assigned, including single task assignment, group task assignment, and emergency task assignment; If the allocation type is group task allocation, obtain the priority of each task in the task group to be allocated; The first task is assigned to a GPU node according to the priority, so that the GPU node generates an affinity label between the first task and the unassigned task. The first task is the highest priority task in the current task group to be assigned, and the affinity label responds to the identity label of the unassigned task. The unassigned tasks are assigned to GPU nodes based on the affinity tags; The step of assigning the unassigned task to the GPU node based on the affinity tag includes: For the unassigned task, perform a first iteration operation, which includes: obtaining a second task, broadcasting the identity tag of the second task to all GPU nodes so that GPU nodes with an affinity tag related to the second task respond to the identity tag of the second task and send the affinity tag, and assigning the second task to a GPU node according to the affinity tag; the second task is the highest priority task among the currently unassigned tasks.

2. The method for allocating computing resources through heterogeneous GPU hybrid scheduling according to claim 1, characterized in that, The step of assigning the second task to a GPU node based on the affinity tag includes: If a hard affinity label exists among the affinity labels, the second task is assigned to the GPU node corresponding to the hard affinity label; If there is no hard affinity label among the affinity labels, but there is a hard anti-affinity label, the second task is assigned to a GPU node other than the GPU node corresponding to the hard anti-affinity label.

3. The method for allocating computing resources through heterogeneous GPU hybrid scheduling according to claim 1, characterized in that, After assigning the second task to a GPU node based on the affinity tag, the computing resource allocation method further includes: The second task is defined as the third task, so that the GPU node generates an affinity label between the third task and the currently unassigned task.

4. The method for allocating computing resources through heterogeneous GPU hybrid scheduling according to claim 1, characterized in that, While broadcasting the identity label of the second task to all GPU nodes, the computing resource allocation method further includes: The first data in the second task is transmitted to GPU nodes within a preset distance, wherein the time for transmitting the first data in the second task to GPU nodes within a preset distance is equal to the time for broadcasting the identity tag of the second task to all GPU nodes, and the first data is the data in the second task that is located at a preset position.

5. The method for allocating computing resources through heterogeneous GPU hybrid scheduling according to claim 4, characterized in that, The step of assigning the second task to a GPU node based on the affinity tag includes: Based on the affinity tag, the second data in the second task is transmitted to the GPU node; Send a transmission command to the GPU node that is closest to the GPU node to which the second task is assigned among the GPU nodes within the preset distance, so that the GPU node sends the first data to the GPU node to which the second task is assigned; A deletion command is sent to GPU nodes within the preset distance to cause the GPU nodes within the preset distance to delete the first data.

6. The method for allocating computing resources through heterogeneous GPU hybrid scheduling according to claim 1, characterized in that, The computing resource allocation method further includes: If the allocation type is a single task allocation, obtain the priority and computing power configuration requirements of the task to be allocated; According to the priority, the task to be assigned is placed into the corresponding priority queue; A second iteration operation is performed on the priority queue. The second iteration operation includes: retrieving a fourth task from the priority queue; determining whether there is a GPU node that meets the computing power configuration requirements of the fourth task and the resource usage of each GPU node based on the computing power configuration requirements of the fourth task and the resource usage of each GPU node; if not, allocating a GPU node to the fourth task according to the retry mechanism and the preemption mechanism, until all tasks in the priority queue are allocated, wherein the fourth task is the task located at the head of the priority queue.

7. The method for allocating computing resources through heterogeneous GPU hybrid scheduling according to claim 6, characterized in that, The allocation of GPU nodes for the fourth task based on the retry and preemption mechanisms includes: The GPU node is allocated to the fourth task according to the preset retry interval. If the fourth task is still not allocated a GPU node when the number of retries reaches the preset number of retries, the retries are stopped. Obtain the execution progress and computing power configuration requirements of the tasks to be preempted; determine the first target task based on the execution progress and computing power configuration requirements; the tasks to be preempted are determined based on the priority and computing power configuration requirements of the currently executing tasks. Pause the first target task and assign the fourth task to the GPU node corresponding to the first target task.

8. The method for allocating computing resources through heterogeneous GPU hybrid scheduling according to claim 1, characterized in that, The computing resource allocation method further includes: If the allocation type is emergency task allocation, obtain the priority of the task to be allocated; Determine the second target task from the currently executing tasks based on the priority; The dependencies of the second target task are determined based on a preset directed acyclic graph. A GPU node is selected from the GPU nodes corresponding to the second target task, and the task to be assigned is assigned to the GPU node.

9. A computing resource allocation system for heterogeneous GPU hybrid scheduling, used to implement the computing resource allocation method for heterogeneous GPU hybrid scheduling as described in any one of claims 1-8, characterized in that, The computing resource allocation system includes a scheduler and multiple GPU nodes, wherein the scheduler is communicatively connected to the multiple GPU nodes, and the multiple GPU nodes are communicatively connected to each other. The scheduler is used to identify the allocation type corresponding to the task to be assigned, which includes single task allocation, group task allocation, and emergency task allocation; if the allocation type is group task allocation, the priority of each task in the task group to be assigned is obtained; and the first task is allocated to a GPU node according to the priority. Unassigned tasks are assigned to GPU nodes based on the affinity tags; The GPU node is used to generate affinity tags for the first task and unassigned tasks. The first task is the highest priority task in the current task group to be assigned. The affinity tag is in response to the identity tag of the unassigned task.

Images