Heterogeneous cluster scheduling method and device, electronic equipment and storage medium

Abstract

The present disclosure relates to a heterogeneous cluster scheduling method and device, electronic equipment and storage medium, the method comprising: obtaining resource occupation of at least one first core satisfying a response condition; sorting each first core based on each resource occupation to obtain a core scheduling sequence; obtaining at least one to-be-completed task submitted by a client, sorting each to-be-completed task based on resource requirements for executing each to-be-completed task to obtain a task execution sequence; determining a target first core corresponding to each to-be-completed task based on the core scheduling sequence and the task execution sequence; and assigning each to-be-completed task to the target first core corresponding thereto. The method of the present disclosure can improve the utilization rate of cluster computing resources and reduce the probability of task blocking. The training and inference process of machine learning can be independent of specific chips, improving the flexibility and portability of machine models and making the load of the cluster more balanced.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a heterogeneous cluster scheduling method and apparatus, electronic device and storage medium. Background Technology

[0002] In today's era of rapid development in information technology, data centers, as the core infrastructure supporting applications such as cloud computing, big data, and artificial intelligence, are becoming increasingly important. Against this backdrop, the study of scheduling issues for heterogeneous clusters, as a crucial component of data centers, is particularly important.

[0003] Heterogeneous clusters, as an effective solution, can provide powerful computing capabilities by integrating different types of computing resources (such as CPUs, GPUs, FPGAs, etc.). However, in practical use, there are still situations where some computing resources are not properly idle and some tasks are blocked, and resource utilization and task processing speed need to be improved.

[0004] Furthermore, heterogeneous machine learning clusters may require hardware with specific processing capabilities to support the training and inference of machine learning models. This can lead to a dependence on specific hardware, limiting the flexibility and portability of models, and making the allocation and management of computing resources more complex; different types of machine learning models may also result in uneven load distribution within the cluster. Summary of the Invention

[0005] In view of this, this disclosure proposes a heterogeneous cluster scheduling scheme.

[0006] According to one aspect of this disclosure, a heterogeneous cluster scheduling method is provided, applied to a software-defined scheduler, comprising: acquiring the resource occupancy status of at least one first kernel that meets the response conditions; sorting the first kernels based on the resource occupancy status to obtain a kernel scheduling order; acquiring at least one task to be completed submitted by a client, sorting the tasks to be completed based on the resource requirements for executing each task to obtain a task execution order; determining the target first kernel corresponding to each task to be completed based on the kernel scheduling order and the task execution order; and assigning each task to be completed to its corresponding target first kernel.

[0007] In one possible implementation, the at least one first kernel includes all kernels in the cluster, the response condition includes responding to the full polling of the scheduler, the resource occupancy includes a first resource occupancy, and obtaining the resource occupancy of the at least one first kernel that meets the response condition includes: when full polling is initiated, obtaining the first resource occupancy of each kernel, the first resource occupancy being data provided by the kernel in response to the full polling of the scheduler; and sorting the first kernels based on their resource occupancy to obtain a kernel scheduling order includes: determining at least one first available kernel from the kernels of the cluster according to their first resource occupancy; determining a first duration required for each first available kernel to reach an idle state; determining a first scheduling order for each first available kernel according to their first duration, and using the first scheduling order as the kernel scheduling order.

[0008] In one possible implementation, the response condition includes: the completion rate of the current task is not less than a completion rate threshold; the at least one first kernel includes a first reserve idle kernel in the cluster whose current task completion rate is not less than the completion rate threshold; the resource occupancy includes a second resource occupancy; obtaining the resource occupancy of at least one first kernel that meets the response condition includes: obtaining the second resource occupancy of each first reserve idle kernel, the second resource occupancy being data provided to the scheduler by each first reserve idle kernel when the current task completion rate is not less than the completion rate threshold; sorting each first kernel based on each resource occupancy to obtain a kernel scheduling order includes: determining the second duration required for each first reserve idle kernel to reach an idle state according to each second resource occupancy; determining the second scheduling order of each first reserve idle kernel according to each second duration, and using the second scheduling order as the kernel scheduling order.

[0009] In one possible implementation, the response conditions include: sending a callable indication to the client, the at least one first kernel including a second reserved idle kernel in the cluster that actively sends the callable indication to the client, the resource occupancy including a third resource occupancy, and obtaining the resource occupancy of at least one first kernel that satisfies the response conditions including: polling each second reserved idle kernel in response to a client's start command to obtain the third resource occupancy of each second reserved idle kernel, the start command being an instruction generated by the client in response to the callable indication, the start command containing information locating the second reserved idle kernel in the cluster; and sorting each first kernel based on each resource occupancy to obtain a kernel scheduling order including: determining the third duration required for each second reserved idle kernel to reach an idle state according to each third resource occupancy; determining the third scheduling order of each second reserved idle kernel according to each third duration, and using the third scheduling order as the kernel scheduling order.

[0010] In one possible implementation, obtaining at least one pending task submitted by the client and sorting the pending tasks based on the resource requirements for executing each pending task to obtain the task execution order includes: when the pending tasks do not include execution priorities, sorting the at least one pending task according to its submission time and resource requirements to obtain the task execution order; and when the pending tasks include execution priorities, sorting the at least one pending task according to its execution priority and resource requirements to obtain the task execution order.

[0011] In one possible implementation, the method further includes: obtaining a first pointer corresponding to each target first kernel, the first pointer pointing to a buffer in a shared space, the same buffer storing the resource occupancy and tasks to be completed for the same first kernel; obtaining the resource occupancy of at least one first kernel that meets the response conditions includes: obtaining the resource occupancy provided by each first kernel from the shared space based on each first pointer; determining the target first kernel corresponding to each task to be completed based on the kernel scheduling order and the task execution order includes: storing the tasks to be completed corresponding to each target first kernel in the buffer indicated by its respective first pointer based on the first pointer corresponding to each target first kernel; and allocating each task to be completed to its respective target first kernel includes: sending a task retrieval instruction to each target first kernel.

[0012] In one possible implementation, the task execution order includes the subtask execution order. The method further includes: finely dividing each task to be completed into multiple subtasks; sorting each task to be completed to obtain the task execution order includes: when the task to be completed does not include execution priority, sorting each subtask according to the submission time of the task to be completed and the connection order of each subtask of the same task to obtain the task execution order; when the task to be completed includes execution priority, the subtask inherits the execution priority of its corresponding task to be completed, and sorting each subtask according to the execution priority and the connection order of each subtask of the same task to obtain the task execution order; determining the target first kernel corresponding to each task to be completed based on the kernel scheduling order and the task execution order includes: matching the first kernels and subtasks with the same order in the kernel scheduling order and task execution order, and taking the first kernel that matches the subtask as the target first kernel for completing the subtask.

[0013] According to another aspect of this disclosure, a heterogeneous cluster scheduling apparatus is provided, comprising:

[0014] The resource usage acquisition unit is used to acquire the resource usage of at least one first kernel that meets the response conditions;

[0015] The kernel scheduling order determination unit is used to sort each of the first kernels based on the resource occupancy status to obtain the kernel scheduling order;

[0016] The task execution order determination unit is used to obtain at least one task to be completed submitted by the client, sort the tasks to be completed based on the resource requirements for executing each task, and obtain the task execution order.

[0017] The target first kernel determination unit is used to determine the target first kernel corresponding to each task to be completed based on the kernel scheduling order and the task execution order.

[0018] The task allocation unit is used to assign each task to be completed to its corresponding target first kernel.

[0019] In one possible implementation, the at least one first kernel includes all kernels within the cluster, the response condition includes responding to a full polling by the scheduler, and the resource occupancy acquisition unit is further configured to:

[0020] When full polling is initiated, the first resource usage of each kernel is obtained. The first resource usage is the data provided by the kernel in response to the scheduler's full polling.

[0021] The kernel scheduling order determination unit is further used for:

[0022] Based on the resource occupancy status of each resource, at least one first available kernel is determined from the kernels of the cluster.

[0023] Determine the first duration required for each first available kernel to reach an idle state;

[0024] According to each of the first durations, determine the first scheduling order of each first available kernel, and use the first scheduling order as the kernel scheduling order.

[0025] In one possible implementation, the response condition includes: the completion rate of the current task is not less than a completion rate threshold; the at least one first kernel includes a first reserved idle kernel in the cluster whose current task completion rate is not less than the completion rate threshold; the resource occupancy status includes a second resource occupancy status; and the resource occupancy status acquisition unit is further configured to:

[0026] Obtain the second resource occupancy status of each first reserve idle kernel. The second resource occupancy status is the data provided to the scheduler by each first reserve idle kernel when the completion rate of the current task is not less than the completion rate threshold.

[0027] The kernel scheduling order determination unit is further used for:

[0028] Based on the usage of each second resource, determine the second duration required for each first reserve idle kernel to reach an idle state;

[0029] According to each of the second durations, the second scheduling order of each first reserved idle kernel is determined, and the second scheduling order is used as the kernel scheduling order.

[0030] In one possible implementation, the response conditions include: sending a callable indication to the client, wherein the at least one first kernel includes a second reserved idle kernel in the cluster that actively sends the callable indication to the client, and the resource occupancy status includes a third resource occupancy status.

[0031] The resource usage acquisition unit is further used for:

[0032] In response to the client's startup command, the system polls each of the second reserve idle kernels to obtain the third resource usage status of each second reserve idle kernel. The startup command is a command generated by the client in response to the callable indication, and the startup command contains information on locating the second reserve idle kernels in the cluster.

[0033] The kernel scheduling order determination unit is further used for:

[0034] Based on the usage of each third resource, determine the third time required for each second reserve idle kernel to reach an idle state;

[0035] According to the third duration, the third scheduling order of each second reserve idle kernel is determined, and the third scheduling order is used as the kernel scheduling order.

[0036] In one possible implementation, the task execution order determination unit is used for:

[0037] If the tasks to be completed do not have execution priorities, the tasks to be completed are sorted according to their submission time and resource requirements to obtain the task execution order.

[0038] If the tasks to be completed include execution priorities, the at least one task to be completed is sorted according to its respective execution priority and resource requirements to obtain the task execution order.

[0039] In one possible implementation, the device includes:

[0040] The first pointer acquisition unit is used to acquire the first pointer corresponding to each target first kernel. The first pointer points to the buffer in the shared space. The same buffer stores the resource usage and tasks to be completed of the same first kernel.

[0041] The resource usage acquisition unit is used for:

[0042] Based on each first pointer, the resource usage provided by each first kernel is obtained from the shared space;

[0043] The target first kernel determination unit is used for:

[0044] Based on the first pointer corresponding to the first kernel of each target, the tasks to be completed corresponding to the first kernel of each target are stored in the buffer indicated by the first pointer of each target.

[0045] The task allocation unit is used for:

[0046] Send task retrieval instructions to the first kernel of each target.

[0047] In one possible implementation, the task execution order includes the subtask execution order, and the apparatus further includes:

[0048] The subtask partitioning unit is used to divide each task to be completed into multiple subtasks in a fine-grained manner.

[0049] The task execution order determination unit is further used for:

[0050] Without considering execution priority, the tasks to be completed are sorted according to their submission time and the connection order of their subtasks to obtain the task execution order.

[0051] When the task to be completed includes an execution priority, the subtasks inherit the execution priority of their respective tasks to be completed. The subtasks are sorted according to their execution priorities and the connection order of the subtasks of the same task to be completed, thus obtaining the task execution order.

[0052] The target first kernel determination unit is further used for:

[0053] The first kernel and subtask with the same order in the kernel scheduling order and task execution order are matched with each other, and the first kernel that matches the subtask is taken as the target first kernel to complete the subtask.

[0054] According to another aspect of this disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the above-described method when executing instructions stored in the memory.

[0055] According to another aspect of this disclosure, a non-volatile computer-readable storage medium is provided that stores computer program instructions thereon, wherein the computer program instructions, when executed by a processor, implement the above-described method.

[0056] According to another aspect of this disclosure, a computer program product is provided, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the above-described method.

[0057] This disclosure abandons the method of obtaining cluster resource usage through hardware schedulers on individual chips. Instead, it establishes a software-based scheduler for the entire cluster, with each chip's kernel directly providing its resource usage information to this scheduler. This breaks down chip limitations, treating the cluster's computing power as a whole. It provides fine-grained consideration of each kernel's resource usage to improve task allocation rationality, increase cluster computing resource utilization, and reduce the probability of task blocking. Furthermore, because of this fine-grained consideration of each kernel, this method does not rely on on-chip hardware schedulers and is not limited by the model or performance of the chips in the cluster. As long as the resource usage meets the requirements of the task to be completed, the kernel can execute the task, which does not depend on a specific chip. In this way, the machine learning training and inference process can be independent of specific chips, improving model flexibility and portability, and resulting in a more balanced cluster load.

[0058] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0059] The accompanying drawings, which are included in and form part of this specification, illustrate exemplary embodiments, features, and aspects of this disclosure together with the specification and serve to explain the principles of this disclosure.

[0060] Figure 1 This is a flowchart illustrating the heterogeneous cluster scheduling method provided in an embodiment of this disclosure.

[0061] Figure 2 This is a schematic diagram of the structure of the heterogeneous cluster scheduling device provided in the embodiments of this disclosure.

[0062] Figure 3 This is a schematic diagram of another heterogeneous cluster scheduling device provided in an embodiment of this disclosure.

[0063] Figure 4 This is a schematic diagram of the structure of an electronic device for heterogeneous cluster scheduling provided in an embodiment of this disclosure. Detailed Implementation

[0064] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0065] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0066] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0067] In today's era of rapid development in information technology, data centers, as the core infrastructure supporting applications such as cloud computing, big data, and artificial intelligence, are becoming increasingly important. Against this backdrop, the study of scheduling issues for heterogeneous clusters, as a crucial component of data centers, is particularly important.

[0068] Heterogeneous clusters allocate tasks based on the chip resource utilization provided by the hardware scheduler on each chip. The hardware scheduler assesses resource utilization across the entire chip as a whole. Therefore, it's possible that some computing resources on a chip may be idle, but the overall resource utilization has reached a threshold, preventing tasks from being assigned to that chip. This results in a situation where there are idle computing resources in the cluster, but tasks still need to wait before allocation can proceed, leading to task blocking. Furthermore, machine learning model inference and training may be dependent on specific hardware (such as a particular chip model), limiting the model's flexibility and portability, and potentially causing uneven load distribution within the cluster.

[0069] Furthermore, clusters typically use the same type of chip. This is because the resource usage data sent by different hardware schedulers is comparable, facilitating task allocation. This further increases the dependence of machine learning models on specific hardware.

[0070] Figure 1 This is a flowchart illustrating a heterogeneous cluster scheduling method provided in an embodiment of this disclosure. The method is applied to a software-defined scheduler. Figure 1 As shown, the method includes:

[0071] S11, obtain the resource usage of at least one first kernel that meets the response conditions.

[0072] A cluster can include multiple kernels, which may be distributed across different or identical chips, or some kernels may be distributed across the same chip. The cluster can also include multiple chips, whose models and performance may be the same or different. In this embodiment, the response condition can be a condition for the kernel to provide the scheduler with its own resource usage information. The kernel and the scheduler can communicate directly. The response condition can be triggered externally to the kernel, and the kernel responds to the trigger. Alternatively, the response condition can be triggered internally to the kernel, and the kernel responds to the trigger. This standardizes the timing of the kernel providing resource usage information, achieving a reasonable frequency that satisfies task allocation needs, reduces task allocation waiting time, and decreases the probability of communication blockage caused by frequent resource usage information provision. Furthermore, the provided resource usage information is more targeted and effective, reducing the probability of obtaining useless information.

[0073] For ease of description, the kernel that meets the response conditions is named the first kernel. Resource usage may include one or more metrics. These metrics, individually or in combination, can characterize the kernel's load. For example, resource usage may include one or more of the following: the proportion of time a single kernel is occupied within a preset time period, the proportion of computing power used by a single kernel to the total computing power of that single kernel, the time difference required to reach an idle state, running speed, and the completion rate of the currently executing task, etc. The resource usage of each first kernel is presented in the same way. That is, the metrics, format, length, and units of the metrics are all the same for each first kernel.

[0074] S12, based on the resource usage, sort each of the first kernels to obtain the kernel scheduling order.

[0075] In this embodiment, the first kernels can be directly sorted according to one or more indicators of resource usage; alternatively, resource usage can be calculated to obtain the result, and the first kernels can be sorted according to the result. This yields the kernel scheduling order. The kernel scheduling order characterizes the order in which each first kernel receives tasks to be completed.

[0076] For example, the higher the idle level of the first core's computing power, the higher its position in the kernel scheduling order.

[0077] For example, the shorter the time it takes for the first kernel to reach an idle state, the higher its position in the kernel scheduling order.

[0078] The above is merely an example demonstrating the case where only one aspect is considered for sorting. Multiple aspects can also be considered for sorting, or sorting can be performed multiple times according to different emphases to obtain multiple kernel scheduling orders. This disclosure does not limit the sorting rules for the first kernel.

[0079] S13: Obtain at least one pending task submitted by the client, sort the pending tasks based on the resource requirements for executing each pending task, and obtain the task execution order.

[0080] The client can respond to external input and obtain tasks to be completed. The client can also execute scripts to retrieve tasks from external sources. The client can communicate directly with the scheduler, submitting tasks to be completed. Tasks to be completed can include operation instructions, operands, etc. The client can submit the task itself to the scheduler, or it can submit the storage address of the task to the scheduler.

[0081] Each task to be completed can include resource requirements, or the scheduler can calculate the corresponding resource requirements based on the tasks to be completed. Each task to be completed can correspond to one resource requirement. Resource requirements can represent the amount of computing resources required from the kernel.

[0082] For example, when the resource requirements of each task to be completed are relatively balanced, the tasks can be sorted according to their submission time to obtain the task execution order.

[0083] For example, the tasks to be completed can be sorted according to resource requirements to obtain the task execution order.

[0084] The above are merely examples illustrating the case where only one aspect is considered for sorting; multiple aspects can also be considered for sorting. This disclosure does not limit the sorting rules for the tasks to be completed.

[0085] S14, based on the kernel scheduling order and the task execution order, determine the target first kernel corresponding to each task to be completed.

[0086] In this embodiment of the disclosure, a task to be completed in the task execution order can be assigned to the first kernel with the same order in the kernel scheduling order. For ease of description, the first kernel that has received the task to be completed can be named the target first kernel of the task to be completed.

[0087] S15 assigns each task to be completed to its corresponding target first kernel.

[0088] The scheduler can send the task to be completed itself to the target first kernel of the task, or it can send the storage address of the task to be completed to the target first kernel.

[0089] This disclosure abandons the method of obtaining cluster resource usage through hardware schedulers on individual chips. Instead, it establishes a software-based scheduler for the entire cluster, with each chip's kernel directly providing its resource usage information to this scheduler. This breaks down chip limitations, treating the cluster's computing power as a whole. It provides fine-grained consideration of each kernel's resource usage to improve task allocation rationality, increase cluster computing resource utilization, and reduce the probability of task blocking. Furthermore, because of this fine-grained consideration of each kernel, this method does not rely on on-chip hardware schedulers and is not limited by the model or performance of the chips in the cluster. As long as the resource usage meets the requirements of the task to be completed, the kernel can execute the task, which does not depend on a specific chip. In this way, the machine learning training and inference process can be independent of specific chips, improving model flexibility and portability, and resulting in a more balanced cluster load.

[0090] In one possible implementation, the at least one first kernel includes all kernels in the cluster, the response condition includes responding to the full polling of the scheduler, the resource occupancy includes a first resource occupancy, and obtaining the resource occupancy of the at least one first kernel that meets the response condition includes: when full polling is initiated, obtaining the first resource occupancy of each kernel, the first resource occupancy being data provided by the kernel in response to the full polling of the scheduler; and sorting the first kernels based on their resource occupancy to obtain a kernel scheduling order includes: determining at least one first available kernel from the kernels of the cluster according to their first resource occupancy; determining a first duration required for each first available kernel to reach an idle state; determining a first scheduling order for each first available kernel according to their first duration, and using the first scheduling order as the kernel scheduling order.

[0091] In this embodiment, the scheduler can poll each kernel in the cluster at a preset polling frequency; for ease of description, this polling is named full polling. For a kernel, the response condition can be a response to the scheduler's full polling. In this way, every kernel will be queried, so every kernel satisfies the response condition. Therefore, in this embodiment, each kernel in the cluster can be the first kernel.

[0092] Each kernel responds to the scheduler's queries (full polling) by providing its own resource usage information to the scheduler. For ease of distinction, the kernel resource usage information in this embodiment will be referred to as the first resource usage information.

[0093] In this embodiment, a first available kernel can be selected based on one or more indicators from the first resource occupancy status. For example, a first available kernel could be a kernel whose completion rate for the currently executing task is higher than a completion rate threshold; or a first available kernel could be a kernel whose resource utilization is lower than a utilization threshold, etc. Then, the first duration of each first available kernel can be determined. If the first duration is one of the indicators of the first resource occupancy status, it can be directly obtained and used. If the first duration is not an indicator of the first resource occupancy status, the first duration can be calculated using the first resource occupancy status. The first available kernels can be sorted according to their first duration, so that the first available kernel with the shorter first duration is allocated to the task to be completed earlier. This yields a first scheduling order for each first available kernel. This first scheduling order is used as the kernel scheduling order. In this embodiment, the kernel scheduling order represents the order in which the first available kernels are scheduled (i.e., the order in which they are allocated to the task to be completed).

[0094] In this embodiment, each kernel only needs to wait for the scheduler to conduct a full poll, and in response to the full poll, provides the scheduler with its initial resource occupancy status. No configuration is required on the kernel side, reducing workload. The scheduler can first select the first available kernel based on the initial resource occupancy status, and then sort them, reducing the sorting workload and balancing the accuracy and efficiency of determining the kernel scheduling order.

[0095] In one possible implementation, the response condition includes: the completion rate of the current task is not less than a completion rate threshold; the at least one first kernel includes a first reserve idle kernel in the cluster whose current task completion rate is not less than the completion rate threshold; the resource occupancy includes a second resource occupancy; obtaining the resource occupancy of at least one first kernel that meets the response condition includes: obtaining the second resource occupancy of each first reserve idle kernel, the second resource occupancy being data provided to the scheduler by each first reserve idle kernel when the current task completion rate is not less than the completion rate threshold; sorting each first kernel based on each resource occupancy to obtain a kernel scheduling order includes: determining the second duration required for each first reserve idle kernel to reach an idle state according to each second resource occupancy; determining the second scheduling order of each first reserve idle kernel according to each second duration, and using the second scheduling order as the kernel scheduling order.

[0096] In this embodiment, each kernel can evaluate the completion rate of its current task at a preset first evaluation frequency. If the completion rate is not less than a completion rate threshold, the kernel can provide its resource usage information to the scheduler. In this embodiment, the first kernel is the kernel whose current task completion rate is not less than the completion rate threshold. For ease of description, in this embodiment, the kernel with a current task completion rate not less than the completion rate threshold can be named the first reserved idle kernel; the resource usage information of the first reserved idle kernel can be named the second resource usage information.

[0097] In this embodiment, if the second duration is one of the indicators of the second resource occupancy, it can be directly obtained and used. If the second duration is not an indicator of the first resource occupancy, it can be calculated using the second resource occupancy. The first reserved idle kernels can be sorted according to their second durations, so that the first reserved idle kernels with shorter second durations are allocated to tasks earlier. This yields a second scheduling order for the first reserved idle kernels. This second scheduling order is used as the kernel scheduling order. In this embodiment, the kernel scheduling order represents the order in which the first reserved idle kernels are scheduled (i.e., the order in which they are allocated to tasks).

[0098] In this embodiment of the disclosure, the scheduler does not need to process the resource occupancy of all kernels, but only needs to process the second resource occupancy of the first reserved idle kernels, which reduces the occupation of storage resources and improves the processing speed, thus improving the efficiency of obtaining the kernel scheduling order.

[0099] In one possible implementation, the response conditions include: sending a callable indication to the client, the at least one first kernel including a second reserved idle kernel in the cluster that actively sends the callable indication to the client, the resource occupancy including a third resource occupancy, and obtaining the resource occupancy of at least one first kernel that satisfies the response conditions including: polling each second reserved idle kernel in response to a client's start command to obtain the third resource occupancy of each second reserved idle kernel, the start command being an instruction generated by the client in response to the callable indication, the start command containing information locating the second reserved idle kernel in the cluster; and sorting each first kernel based on each resource occupancy to obtain a kernel scheduling order including: determining the third duration required for each second reserved idle kernel to reach an idle state according to each third resource occupancy; determining the third scheduling order of each second reserved idle kernel according to each third duration, and using the third scheduling order as the kernel scheduling order.

[0100] In this embodiment, each kernel can evaluate the completion rate of its current task or the waiting time required to reach an idle state according to a preset second evaluation frequency. For ease of description, kernels with a current task completion rate not less than a completion rate threshold can be named second ready-to-idle kernels. Second ready-to-idle kernels can send a callable indication to the client. The client can visually inform external devices or users that a kernel is about to become idle and that a task to be completed can be entered.

[0101] The client can generate a startup command to initiate the scheduler to poll all second-stage idle kernels, provided that the command generation conditions are met. This embodiment of the disclosure does not limit the generation conditions. For example, the generation condition could be that the client has received a first number of callable indications, or that the client has received callable indications for a fourth duration. The startup command may include information that allows the scheduler to locate the second-stage idle kernels in the cluster; for example, the startup command may include a kernel identifier. In response to the startup command, the scheduler polls each second-stage idle kernel. Each second-stage idle kernel, in response to the scheduler's polling, provides its resource usage status to the scheduler. For ease of distinction, the resource usage status of the second-stage idle kernels in this initial embodiment is referred to as the third resource usage status.

[0102] In this embodiment, if the third duration is one of the indicators of the third resource occupancy, it can be directly obtained and used. If the third duration is not an indicator of the third resource occupancy, it can be calculated using the third resource occupancy. The second reserved idle kernels can be sorted according to their third durations, so that the first reserved idle kernel with the shorter third duration is allocated to the task to be completed earlier. This yields the third scheduling order of the second reserved idle kernels. This third scheduling order is used as the kernel scheduling order. In this embodiment, the kernel scheduling order represents the order in which the third reserved idle kernels are scheduled (i.e., the order in which they are allocated to the task to be completed).

[0103] In this embodiment, the second reserved idle kernel directly sends a callable indication to the client, which instructs the scheduler to poll the second reserved idle kernel. This buys time to acquire tasks and shortens the waiting time. The scheduler can obtain the status of tasks and resource occupancy in near-synchronous fashion, reducing the idle rate of the second reserved idle kernel and improving kernel utilization.

[0104] In one possible implementation, obtaining at least one pending task submitted by the client and sorting the pending tasks based on the resource requirements for executing each pending task to obtain the task execution order includes: when the pending tasks do not include execution priorities, sorting the at least one pending task according to its submission time and resource requirements to obtain the task execution order; and when the pending tasks include execution priorities, sorting the at least one pending task according to its execution priority and resource requirements to obtain the task execution order.

[0105] In this embodiment of the disclosure, the task to be completed may include the task itself, and may also include the submission time, and / or the required resources, and / or the execution priority.

[0106] In this embodiment, users can submit tasks on the client according to their urgency. The tasks can then be sorted based on submission time and resource requirements, ensuring that tasks with the same resource requirements are assigned to the first target kernel first. Alternatively, multiple clients may submit tasks at the same time. In this case, the tasks need to include execution priorities. The tasks can then be sorted based on execution priority and resource requirements, ensuring that tasks with the same resource requirements are assigned to the first target kernel first.

[0107] Previously, when using a chip hardware scheduler for task allocation, only one strategy could be used for scheduling. However, the method of this disclosure makes kernel scheduling more flexible and applicable to a wider range of scenarios.

[0108] In one possible implementation, the method further includes: obtaining a first pointer corresponding to each target first kernel, the first pointer pointing to a buffer in a shared space, the same buffer storing the resource occupancy and tasks to be completed for the same first kernel; obtaining the resource occupancy of at least one first kernel that meets the response conditions includes: obtaining the resource occupancy provided by each first kernel from the shared space based on each first pointer; determining the target first kernel corresponding to each task to be completed based on the kernel scheduling order and the task execution order includes: storing the tasks to be completed corresponding to each target first kernel in the buffer indicated by its respective first pointer based on the first pointer corresponding to each target first kernel; and allocating each task to be completed to its respective target first kernel includes: sending a task retrieval instruction to each target first kernel.

[0109] The shared space is storage space. It can include multiple buffers, each corresponding to a kernel. Each buffer can store the resource usage of its corresponding kernel and the tasks to be completed allocated to that kernel. The scheduler can store tasks to be completed in the buffer, and the kernel retrieves these tasks from the buffer; similarly, the kernel can store resource usage in the buffer, and the scheduler retrieves resource allocation information from the buffer. Resource usage and tasks to be completed are not stored simultaneously; therefore, the same buffer can be used to alternately store the resource usage of the corresponding kernel and the tasks to be completed, saving storage space and improving storage utilization.

[0110] Each buffer corresponds to a first pointer, thus establishing a direct correspondence between the buffer, kernel, and first pointer. The target first kernel stores its resource usage in a shared-space buffer. Each target first kernel can correspond to one buffer. The target first kernel sends the first pointer of its corresponding buffer to the scheduler. The scheduler retrieves the resource usage from the shared-space buffer based on the first pointer. After allocating a target first kernel to each task, the task is stored in the buffer corresponding to the matched target first kernel. A task retrieval command is sent to the matched target first kernel. The target first kernel responds to the task retrieval command by using the first pointer of its corresponding buffer to retrieve the task from its own buffer. The kernel and resource scheduler do not need to transmit resource usage information or tasks; only the first pointer and the task retrieval command are transmitted, reducing the amount of data transmission between the kernel and the resource scheduler and saving communication resources.

[0111] In one possible implementation, the task execution order includes the subtask execution order. The method further includes: finely dividing each task to be completed into multiple subtasks; sorting each task to be completed to obtain the task execution order includes: when the task to be completed does not include execution priority, sorting each subtask according to the submission time of the task to be completed and the connection order of each subtask of the same task to obtain the task execution order; when the task to be completed includes execution priority, the subtask inherits the execution priority of its corresponding task to be completed, and sorting each subtask according to the execution priority and the connection order of each subtask of the same task to obtain the task execution order; determining the target first kernel corresponding to each task to be completed based on the kernel scheduling order and the task execution order includes: matching the first kernels and subtasks with the same order in the kernel scheduling order and task execution order, and taking the first kernel that matches the subtask as the target first kernel for completing the subtask.

[0112] The scheduler can also split pending tasks into subtasks. A pending task can correspond to two or more subtasks. Subtasks can have a sequential order, which characterizes their execution order. Furthermore, the sequential order characterizes the flow of intermediate data generated by each subtask. When a pending task does not have an execution priority, it can be sorted according to its submission time and the sequential order of its corresponding subtasks. When a pending task has an execution priority, its subtasks can inherit that priority. Therefore, subtasks can be sorted according to their execution priority and the sequential order of their corresponding subtasks, and the execution order of the subtasks will be used as the task execution order.

[0113] In this embodiment, the task to be completed is subdivided and the subtasks are sorted. The task blocks are allocated to the target first core for execution, which reduces the probability of task blocking and allows for more refined utilization of computing resources in the cluster, thereby improving the utilization rate of computing resources in the cluster.

[0114] In one possible implementation, the resource usage of the first kernel includes: the utilization rate and latency of the streaming multiprocessors within the first kernel, etc. The compilers of each chip in the cluster can be modified to ensure that the resource usage metrics of each first kernel are identical in length and format.

[0115] Figure 2This is a schematic diagram of the heterogeneous cluster scheduling device provided in an embodiment of this disclosure. The device includes n clients and m cores, where n and m are positive integers. These cores can be cores on multiple chips within the cluster. The chips in the cluster can be central processing units (CPUs), graphics processing units (GPUs), field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), etc., and this disclosure does not limit the specific model and type of the chips. The device also includes a software-defined scheduler and a shared space for storing data. The scheduler may include a resource extraction unit, a resource management unit, and a task planning unit.

[0116] Each kernel can communicate with each client; each kernel can communicate with the scheduler, and each client can communicate with the scheduler. Both the scheduler and the kernel can access and retrieve data from the shared space.

[0117] In this embodiment, each kernel can evaluate the completion rate of its current task at a preset second evaluation frequency. A kernel with a completion rate of not less than 95% can be designated as a second reserve idle kernel. The second reserve idle kernel can send a callable indication to the client. The client generates a startup instruction if the instruction generation conditions are met and sends the startup instruction to the scheduler. In response to the startup instruction, the scheduler polls each of the second reserve idle kernels. In response to the scheduler's polling, the second reserve idle kernel stores the third resource occupancy status in a buffer in the shared space. Figure 2 (Buffers are not shown in the diagram). Each second-stage spare idle kernel sends a first pointer to its corresponding buffer to the scheduler. The scheduler can use the first pointer to obtain the third resource occupancy status of each second-stage spare idle kernel from the shared space. The client sends its tasks to be completed to the scheduler. The resource extraction unit extracts available information from the third resource occupancy status. Based on the available information, the resource management unit sorts the second-stage spare idle kernels to obtain the kernel scheduling order. The task planning unit sorts the tasks to be completed to obtain the task execution order, and based on the kernel scheduling order and task execution order, matches the tasks to be completed with each second-stage spare idle kernel to determine the target first kernel for each task to be completed. Based on the first pointer of the buffer corresponding to the target first kernel, the scheduler stores the tasks to be completed in the buffer corresponding to the target first kernel and sends a task retrieval instruction to each target first kernel. Each target first kernel retrieves the tasks to be completed from the shared space and executes them based on the first pointer of its corresponding buffer.

[0118] Figure 3 A schematic diagram of another heterogeneous cluster scheduling device provided in an embodiment of this disclosure. The device 20 includes:

[0119] Resource occupancy acquisition unit 21 is used to acquire the resource occupancy of at least one first kernel that meets the response conditions;

[0120] The kernel scheduling order determination unit 22 is used to sort each of the first kernels based on the resource occupancy status to obtain the kernel scheduling order;

[0121] The task execution order determination unit 23 is used to obtain at least one task to be completed submitted by the client, sort the tasks to be completed based on the resource requirements for executing each task, and obtain the task execution order.

[0122] The target first kernel determination unit 24 is used to determine the target first kernel corresponding to each task to be completed based on the kernel scheduling order and the task execution order.

[0123] The task allocation unit 25 is used to allocate each task to be completed to its corresponding target first kernel.

[0124] In one possible implementation, the at least one first kernel includes all kernels within the cluster, the response condition includes responding to a full polling by the scheduler, and the resource occupancy acquisition unit 21 is further configured to:

[0125] When full polling is initiated, the first resource usage of each kernel is obtained. The first resource usage is the data provided by the kernel in response to the scheduler's full polling.

[0126] The kernel scheduling order determination unit 22 is further used for:

[0127] Based on the resource occupancy status of each resource, at least one first available kernel is determined from the kernels of the cluster.

[0128] Determine the first duration required for each first available kernel to reach an idle state;

[0129] According to each of the first durations, determine the first scheduling order of each first available kernel, and use the first scheduling order as the kernel scheduling order.

[0130] In one possible implementation, the response condition includes: the completion rate of the current task is not less than a completion rate threshold; the at least one first kernel includes a first reserved idle kernel in the cluster whose current task completion rate is not less than the completion rate threshold; the resource occupancy status includes a second resource occupancy status; and the resource occupancy status acquisition unit 21 is further configured to:

[0131] Obtain the second resource occupancy status of each first reserve idle kernel. The second resource occupancy status is the data provided to the scheduler by each first reserve idle kernel when the completion rate of the current task is not less than the completion rate threshold.

[0132] The kernel scheduling order determination unit 22 is further used for:

[0133] Based on the usage of each second resource, determine the second duration required for each first reserve idle kernel to reach an idle state;

[0134] According to each of the second durations, the second scheduling order of each first reserved idle kernel is determined, and the second scheduling order is used as the kernel scheduling order.

[0135] In one possible implementation, the response conditions include: sending a callable indication to the client, wherein the at least one first kernel includes a second reserved idle kernel in the cluster that actively sends the callable indication to the client, and the resource occupancy status includes a third resource occupancy status.

[0136] The resource occupancy acquisition unit 21 is further used for:

[0137] In response to the client's startup command, the system polls each of the second reserve idle kernels to obtain the third resource usage status of each second reserve idle kernel. The startup command is a command generated by the client in response to the callable indication, and the startup command contains information on locating the second reserve idle kernels in the cluster.

[0138] The kernel scheduling order determination unit 22 is further used for:

[0139] Based on the usage of each third resource, determine the third time required for each second reserve idle kernel to reach an idle state;

[0140] According to the third duration, the third scheduling order of each second reserve idle kernel is determined, and the third scheduling order is used as the kernel scheduling order.

[0141] In one possible implementation, the task execution order determination unit 23 is used to:

[0142] If the tasks to be completed do not have execution priorities, the tasks to be completed are sorted according to their submission time and resource requirements to obtain the task execution order.

[0143] If the tasks to be completed include execution priorities, the at least one task to be completed is sorted according to its respective execution priority and resource requirements to obtain the task execution order.

[0144] In one possible implementation, the device 20 includes:

[0145] The first pointer acquisition unit is used to acquire the first pointer corresponding to each target first kernel. The first pointer points to the buffer in the shared space. The same buffer stores the resource usage and tasks to be completed of the same first kernel.

[0146] The resource usage acquisition unit 21 is used for:

[0147] Based on each first pointer, the resource usage provided by each first kernel is obtained from the shared space;

[0148] The target first kernel determination unit 24 is used for:

[0149] Based on the first pointer corresponding to the first kernel of each target, the tasks to be completed corresponding to the first kernel of each target are stored in the buffer indicated by the first pointer of each target.

[0150] The task allocation unit 25 is used for:

[0151] Send task retrieval instructions to the first kernel of each target.

[0152] In one possible implementation, the task execution order includes the subtask execution order, and the device 20 further includes:

[0153] The subtask partitioning unit is used to divide each task to be completed into multiple subtasks in a fine-grained manner.

[0154] The task execution order determination unit 23 is further configured to:

[0155] Without considering execution priority, the tasks to be completed are sorted according to their submission time and the connection order of their subtasks to obtain the task execution order.

[0156] When the task to be completed includes an execution priority, the subtasks inherit the execution priority of their respective tasks to be completed. The subtasks are sorted according to their execution priorities and the connection order of the subtasks of the same task to be completed, thus obtaining the task execution order.

[0157] The target first kernel determination unit 24 is further used for:

[0158] The first kernel and subtask with the same order in the kernel scheduling order and task execution order are matched with each other, and the first kernel that matches the subtask is taken as the target first kernel to complete the subtask.

[0159] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0160] This disclosure also proposes a computer-readable storage medium storing computer program instructions that, when executed by a processor, implement the above-described method. The computer-readable storage medium can be volatile or non-volatile.

[0161] This disclosure also proposes an electronic device, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0162] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the above-described method.

[0163] Figure 4 This is a schematic diagram of the structure of an electronic device for heterogeneous cluster scheduling provided in an embodiment of this disclosure. For example, the electronic device 1900 can be provided as a server or a terminal device. (Refer to...) Figure 4 The electronic device 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by the processing component 1922. The application programs stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the methods described above.

[0164] Electronic device 1900 may also include a power supply component 1926 configured to perform power management of electronic device 1900, a wired or wireless network interface 1950 configured to connect electronic device 1900 to a network, and an input / output interface 1958 (I / O interface). Electronic device 1900 can operate on an operating system, such as Windows Server, stored in memory 1932. TM Mac OS X TM Unix TM LinuxTM FreeBSD TM Or similar.

[0165] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by a processing component 1922 of an electronic device 1900 to perform the above-described method.

[0166] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0167] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0168] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0169] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0170] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0171] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0172] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0173] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0174] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A heterogeneous cluster scheduling method, characterized in that, Software-defined schedulers include: Obtain the resource usage of at least one first kernel that meets the response conditions; Based on the resource usage, the first kernels are sorted to obtain the kernel scheduling order; Get at least one pending task submitted by the client, sort the pending tasks based on the resource requirements for executing each pending task, and obtain the task execution order; Based on the kernel scheduling order and the task execution order, the target first kernel corresponding to each task to be completed is determined; Assign each task to be completed to its corresponding target first kernel; The response conditions include: the completion rate of the current task is not less than a completion rate threshold; the at least one first kernel includes a first reserved idle kernel in the cluster whose current task completion rate is not less than the completion rate threshold; the resource usage includes a second resource usage; and obtaining the resource usage of at least one first kernel that meets the response conditions includes: Obtain the second resource occupancy status of each first reserve idle kernel. The second resource occupancy status is the data provided to the scheduler by each first reserve idle kernel when the completion rate of the current task is not less than the completion rate threshold. The step of sorting each of the first kernels based on their resource usage to obtain the kernel scheduling order includes: Based on the usage of each second resource, determine the second duration required for each first reserve idle kernel to reach an idle state; According to each of the second durations, the second scheduling order of each first reserved idle kernel is determined, and the second scheduling order is used as the kernel scheduling order.

2. The method according to claim 1, characterized in that, The at least one first kernel includes all kernels within the cluster, the response condition includes responding to a full polling by the scheduler, the resource usage includes a first resource usage, and obtaining the resource usage of at least one first kernel that meets the response condition includes: When full polling is initiated, the first resource usage of each kernel is obtained. The first resource usage is the data provided by the kernel in response to the scheduler's full polling. The step of sorting each of the first kernels based on their resource usage to obtain the kernel scheduling order includes: Based on the resource occupancy status of each resource, at least one first available kernel is determined from the kernels of the cluster. Determine the first duration required for each first available kernel to reach an idle state; According to each of the first durations, determine the first scheduling order of each first available kernel, and use the first scheduling order as the kernel scheduling order.

3. The method according to claim 1, characterized in that, The response conditions include: sending a callable indication to the client; the at least one first kernel includes a second reserved idle kernel in the cluster that actively sends the callable indication to the client; and the resource occupancy status includes a third resource occupancy status. The step of obtaining the resource usage of at least one first kernel that meets the response conditions includes: In response to the client's startup command, the system polls each of the second reserve idle kernels to obtain the third resource usage status of each second reserve idle kernel. The startup command is a command generated by the client in response to the callable indication, and the startup command contains information on locating the second reserve idle kernels in the cluster. The step of sorting each of the first kernels based on their resource usage to obtain the kernel scheduling order includes: Based on the usage of each third resource, determine the third time required for each second reserve idle kernel to reach an idle state; According to the third duration, the third scheduling order of each second reserve idle kernel is determined, and the third scheduling order is used as the kernel scheduling order.

4. The method according to claim 1, characterized in that, The process of obtaining at least one pending task submitted by the client, sorting the pending tasks based on their resource requirements to obtain the task execution order, includes: If the tasks to be completed do not have execution priorities, the tasks to be completed are sorted according to their submission time and resource requirements to obtain the task execution order. If the tasks to be completed include execution priorities, the at least one task to be completed is sorted according to its respective execution priority and resource requirements to obtain the task execution order.

5. The method according to claim 1, characterized in that, The method further includes: Obtain the first pointer corresponding to each target's first kernel. The first pointer points to a buffer in the shared space. The same buffer stores the resource usage and tasks to be completed for the same first kernel. The step of obtaining the resource usage of at least one first kernel that meets the response conditions includes: Based on each first pointer, the resource usage provided by each first kernel is obtained from the shared space; The step of determining the target first kernel corresponding to each task to be completed based on the kernel scheduling order and the task execution order includes: Based on the first pointer corresponding to the first kernel of each target, the tasks to be completed corresponding to the first kernel of each target are stored in the buffer indicated by the first pointer of each target. The process of assigning each task to its corresponding target first kernel includes: Send task retrieval instructions to the first kernel of each target.

6. The method according to claim 1, characterized in that, The task execution order includes the subtask execution order, and the method further includes: Each task to be completed is further divided into multiple sub-tasks; The process of sorting the tasks to be completed to obtain the task execution order includes: If the tasks to be completed do not include execution priority, the subtasks are sorted according to their submission time and the connection order of the subtasks of the same task to be completed, thus obtaining the task execution order. When the task to be completed includes an execution priority, the subtasks inherit the execution priority of their respective tasks to be completed. The subtasks are sorted according to their execution priorities and the connection order of the subtasks of the same task to be completed, thus obtaining the task execution order. The step of determining the target first kernel corresponding to each task to be completed based on the kernel scheduling order and the task execution order includes: The first kernel and subtask with the same order in the kernel scheduling order and task execution order are matched with each other, and the first kernel that matches the subtask is taken as the target first kernel to complete the subtask.

7. A heterogeneous cluster scheduling device, characterized in that, include: The resource usage acquisition unit is used to acquire the resource usage of at least one first kernel that meets the response conditions; The kernel scheduling order determination unit is used to sort each of the first kernels based on the resource occupancy status to obtain the kernel scheduling order; The task execution order determination unit is used to obtain at least one task to be completed submitted by the client, sort the tasks to be completed based on the resource requirements for executing each task, and obtain the task execution order. The target first kernel determination unit is used to determine the target first kernel corresponding to each task to be completed based on the kernel scheduling order and the task execution order. The task allocation unit is used to assign each task to be completed to its corresponding target first kernel; The response conditions include: the completion rate of the current task is not less than a completion rate threshold; the at least one first kernel includes a first reserved idle kernel in the cluster whose current task completion rate is not less than the completion rate threshold; the resource usage status includes a second resource usage status; and the resource usage status acquisition unit is further used for: Obtain the second resource occupancy status of each first reserve idle kernel. The second resource occupancy status is the data provided to the scheduler by each first reserve idle kernel when the completion rate of the current task is not less than the completion rate threshold. The kernel scheduling order determination unit is further used for: Based on the usage of each second resource, determine the second duration required for each first reserve idle kernel to reach an idle state; According to each of the second durations, the second scheduling order of each first reserved idle kernel is determined, and the second scheduling order is used as the kernel scheduling order.

8. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to implement the method of any one of claims 1 to 6 when executing instructions stored in the memory.

9. A non-volatile computer-readable storage medium storing computer program instructions thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 6.

Images