GPU computing resource scheduling method, system, device and medium

Abstract

This application provides a GPU computing resource scheduling method, system, device, and medium. The method acquires task information for at least one task to be scheduled and constructs a corresponding task profile based on the task information. It also acquires resource status information for each computing node in the GPU cluster and constructs a corresponding node status based on the resource status information. Then, based on the task profile and node status, it determines the scheduling priority of the task to be scheduled. Finally, based on the task profile, node status, and scheduling priority, it determines a target GPU resource combination for executing the task to be scheduled. Thus, under conditions of large-scale GPU cluster operation, diverse task types, and dynamically changing resource requirements, it selects a target GPU resource combination that can improve overall resource utilization, reduce inter-task interference, and shorten task completion time while ensuring timely execution of high-priority tasks.

Description

Technical Field

[0001] This application relates to computing power scheduling technology, and more particularly to a GPU computing power resource scheduling method, system, device and medium. Background Technology

[0002] Existing GPU cluster scheduling typically employs a coarse-grained strategy based on queue order or priority. Scheduling decisions are primarily based on limited information such as task submission time, static resource requests, and the number of remaining video memory or idle GPUs on a node.

[0003] Resource scheduling decisions based on the aforementioned coarse parameters often result in high-priority tasks queuing up, low actual GPU utilization, and critical tasks being assigned to nodes with unsuitable topologies or severe resource fragmentation in real-world large-scale multi-tenant cluster scenarios. This leads to overall wasted computing power and unstable task completion times. Summary of the Invention

[0004] This application provides a GPU computing resource scheduling method, system, device, and medium to solve the technical problems of overall computing power waste and unstable task completion time caused by resource scheduling judgment based on coarse parameters.

[0005] Firstly, this application provides a GPU computing resource scheduling method, including:

[0006] Obtain task information for at least one task to be scheduled, and construct a corresponding task profile based on the task information;

[0007] Obtain the resource status information of each computing node in the GPU cluster, and construct the corresponding node status based on the resource status information;

[0008] Based on the task profile and the node status, determine the scheduling priority of the task to be scheduled;

[0009] Based on the task profile, the node status, and the scheduling priority, a target GPU resource combination is determined for executing the scheduled task.

[0010] Secondly, this application provides a GPU computing resource scheduling system, comprising:

[0011] The task information acquisition module is used to acquire task information of at least one task to be scheduled, and to construct a corresponding task profile based on the task information.

[0012] The status information acquisition module is used to acquire the resource status information of each computing node in the GPU cluster, and construct the corresponding node status based on the resource status information;

[0013] The priority determination module is used to determine the scheduling priority of the task to be scheduled based on the task profile and the node status.

[0014] The matching score determination module is used to determine the matching score of the task to be scheduled and the target GPU resource combination based on the task profile, the node status and the scheduling priority.

[0015] The resource scheduling module is used to schedule the task to be scheduled to the corresponding target GPU resource combination for execution based on the matching score.

[0016] Thirdly, this application provides an electronic device, comprising:

[0017] Processor; and,

[0018] Memory for storing the executable instructions of the processor;

[0019] The processor is configured to perform any of the possible methods described in the first aspect by executing the executable instructions.

[0020] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement any of the possible methods described in the first aspect.

[0021] The GPU computing resource scheduling method, system, device, and medium provided in this application acquire task information of at least one task to be scheduled and construct a corresponding task profile based on the task information. They also acquire resource status information of each computing node in the GPU cluster and construct a corresponding node status based on the resource status information. Then, based on the task profile and node status, they determine the scheduling priority of the task to be scheduled. Finally, based on the task profile, node status, and scheduling priority, they determine the target GPU resource combination for executing the task to be scheduled. Thus, under conditions of large-scale GPU cluster operation, diverse task types, and dynamically changing resource requirements, they construct task profiles and node statuses that accurately reflect task resource requirements and cluster resource status before scheduling decisions. Based on this, they quantify the matching degree between the scheduling priority of the task to be scheduled and the candidate GPU resource combination, thereby selecting the target GPU resource combination that can improve overall resource utilization, reduce inter-task interference, and shorten task completion time while ensuring the timely execution of high-priority tasks. Attached Figure Description

[0022] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0023] Figure 1 This is a flowchart illustrating a GPU computing resource scheduling method according to an example embodiment of this application;

[0024] Figure 2 This is a flowchart illustrating the task profile construction method in S110 according to an example embodiment of this application;

[0025] Figure 3 This is a flowchart illustrating the node state construction method in S120 according to an example embodiment of this application;

[0026] Figure 4 This is a flowchart illustrating a specific implementation of S130 according to an example embodiment of this application;

[0027] Figure 5 This is a flowchart illustrating a specific implementation of S140 according to an example embodiment of this application;

[0028] Figure 6 This is a flowchart illustrating a GPU computing resource scheduling method according to another example embodiment of this application;

[0029] Figure 7 This is a flowchart illustrating a specific implementation of S230 according to an example embodiment of this application;

[0030] Figure 8 This is a flowchart illustrating a GPU computing resource scheduling method according to yet another example embodiment of this application;

[0031] Figure 9 This is a flowchart illustrating a GPU computing resource scheduling method according to yet another example embodiment of this application;

[0032] Figure 10 This is a schematic diagram of the structure of a GPU computing resource scheduling system according to an example embodiment of this application;

[0033] Figure 11 This is a schematic diagram of the structure of an electronic device according to an example embodiment of this application.

[0034] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0035] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0036] Figure 1 This is a flowchart illustrating a GPU computing resource scheduling method according to an example embodiment of this application. Figure 1 As shown, the method provided in this embodiment includes:

[0037] S110. Obtain task information for at least one task to be scheduled, and construct a corresponding task profile based on the task information.

[0038] In this step, at least one task submission request for a task to be scheduled can be received from the task submission system via a preset interface. The task submission request includes task identification information and a task configuration file corresponding to the task to be scheduled. Then, based on the task identification information, the task metadata information corresponding to the task to be scheduled is retrieved from the task metadata storage. The task metadata information includes at least one of the following: user information to which the task belongs, project or business line information to which the task belongs, and historical execution record identification information. Next, the task configuration file is parsed to obtain at least one of the following: resource requirement configuration, runtime environment configuration, and scheduling constraint configuration corresponding to the task to be scheduled. Finally, the task identification information, task metadata information, and configuration information in the task configuration file are structured and organized to generate the original task information corresponding to the task to be scheduled.

[0039] Furthermore, Figure 2 This is a schematic flowchart illustrating the task profile construction method in S110 according to an example embodiment of this application. Figure 2 As shown, the task profile construction method in S110 above includes:

[0040] S111. Parse the task configuration of the task to be scheduled and obtain task type information, computing power requirement information and timeliness requirement information.

[0041] First, configuration content in a preset format can be read from the task configuration file associated with the task to be scheduled, wherein the task configuration file is at least one of a text configuration file, a structured configuration file, or a database record.

[0042] Secondly, the task configuration file is parsed based on the configuration key name and hierarchical structure to extract configuration fields related to task type, resource requirements and timeliness constraints, forming intermediate configuration data.

[0043] In the intermediate configuration data, the task type information of the task to be scheduled is determined based on the task entry script type, the type of deep learning framework used, and whether there is a distributed training configuration. The task type information includes at least one of the following: training task, inference task, data preprocessing task, and model evaluation task.

[0044] In the intermediate configuration data, configuration fields related to the number of GPUs, single-card memory requirements, desired GPU model, central processing unit core requirements, and memory requirements are parsed to determine the computing power requirements of the task to be scheduled. The computing power requirements information is used to characterize the basic resource requirements of the task to be scheduled on the target GPU resource combination.

[0045] In the intermediate configuration data, configuration fields related to task priority, latest completion time, expected start time, service level target, and whether degraded operation is allowed are parsed to determine the timeliness requirements of the task to be scheduled. The timeliness requirements are used to characterize the constraints of the task to be scheduled on completion time and response latency.

[0046] Then, the task type information, computing power requirement information, and timeliness requirement information are associated with the task identifier of the task to be scheduled and stored for use in subsequent task profile generation.

[0047] S112. Before scheduling tasks are executed, a pre-run analysis is performed, and GPU performance metrics and I / O performance metrics are collected.

[0048] Optionally, pre-running GPU resources can be allocated to the scheduled task from a preset GPU resource pool. Then, during the pre-running phase of the scheduled task, memory usage and computational load information are collected. Next, based on the memory usage and computational load information, the peak memory utilization, average utilization, and computational intensity characteristics of the scheduled task are determined. Finally, the peak memory utilization, average utilization, and computational intensity characteristics are written into the task profile.

[0049] Optional GPU performance metrics include at least one of the following: memory usage curve, stream processor utilization, dedicated matrix operation unit utilization, thread bundle execution efficiency, instruction type distribution, global memory bandwidth utilization, cache hit rate, and GPU interconnect bandwidth utilization.

[0050] Optionally, I / O performance metrics include at least one of disk I / O throughput and network I / O throughput.

[0051] Furthermore, the determination of the aforementioned peak memory utilization, average utilization, and computational density characteristics can specifically include:

[0052] The video memory usage information collected within a preset time period is organized according to the sampling time order to obtain a video memory usage time series. Then, the maximum value of the video memory usage value corresponding to each sampling time is selected from the video memory usage time series as the video memory peak value of the task to be scheduled.

[0053] The memory usage values ​​corresponding to each sampling time in the memory usage time series are accumulated and divided by the product of the number of sampling points and the total memory capacity of a single GPU to obtain the average memory utilization of the task to be scheduled.

[0054] The computing load information collected within a preset time period is normalized to obtain a normalized time series of computing load, wherein the computing load information includes at least one of stream processor utilization and dedicated matrix operation unit utilization.

[0055] Based on the ratio and correlation characteristics between the normalized time series of computing load and the corresponding time series of video memory usage, the computational density characteristics of the task to be scheduled are determined, and the computational density characteristics, along with the peak video memory utilization and the average video memory utilization, are written into the task profile.

[0056] S113. Generate a task profile corresponding to the task to be scheduled based on the task configuration, GPU performance metrics, and I / O performance metrics.

[0057] Based on the task type information, computing power requirement information, and timeliness requirement information obtained from the task configuration, basic configuration information corresponding to the task to be scheduled is constructed. The basic configuration information includes at least one of the following: task type, expected number of GPUs, expected single-card memory requirement, expected GPU model, and timeliness constraint parameters.

[0058] Based on GPU performance metrics, statistical analysis was performed on the memory usage curves collected during the pre-run analysis phase to obtain the memory usage time series, peak memory utilization, average memory utilization, and memory usage fluctuation of the scheduled tasks.

[0059] Based on GPU performance metrics, feature extraction is performed on the stream processor utilization, dedicated matrix operation unit utilization, thread bundle execution efficiency, and instruction type distribution collected during the pre-run analysis phase to determine the computational feature information of the task to be scheduled. The computational feature information includes at least one of computational intensity, matrix operation dependency, and instruction type preference.

[0060] Based on GPU performance metrics, a correlation analysis is performed on global memory bandwidth utilization, cache hit rate, and inter-GPU interconnect bandwidth utilization to determine the data access pattern characteristics of the tasks to be scheduled. The data access pattern characteristics include at least one of bandwidth sensitivity, cache friendliness, and cross-GPU communication strength.

[0061] Based on I / O performance metrics, statistical analysis is performed on disk I / O throughput and network I / O throughput to determine the I / O characteristics of the tasks to be scheduled. The I / O characteristics include at least one of disk I / O intensity and network I / O intensity.

[0062] Based on the time series of video memory usage, computational characteristics, data access pattern characteristics, and I / O characteristics, the phased execution time of the task to be scheduled during the pre-run analysis phase is fitted with the corresponding resource usage to determine the resource usage pattern of the task to be scheduled. The resource usage pattern is used to characterize the correspondence between the time segments of the task to be scheduled during execution and the video memory usage, computational load, and I / O load.

[0063] Based on basic configuration information, memory usage time series, peak memory utilization, average memory utilization, computational characteristics, data access pattern characteristics, I / O characteristics, and resource usage patterns, a task feature vector corresponding to the task to be scheduled is constructed. The task feature vector is used to participate in affinity calculation, interference calculation, and determination of resource adaptation weights in the subsequent scheduling process.

[0064] The basic configuration information, task feature vector, and task identifier corresponding to the task to be scheduled are associated and stored to form a task profile of the task to be scheduled.

[0065] S120. Obtain the resource status information of each computing node in the GPU cluster, and construct the corresponding node status based on the resource status information.

[0066] In this step, the cluster management component can send resource status reporting instructions to each compute node in the GPU cluster at a preset polling cycle. The resource status reporting instructions are used to trigger the corresponding compute node to collect the running status of local GPU resources and central processing unit resources.

[0067] Specifically, on each computing node, the node agent program calls the GPU driver interface and system monitoring interface to obtain at least one of the following raw data: raw data of GPU memory usage, raw data of stream processor utilization, raw data of dedicated matrix operation unit utilization, raw data of temperature, raw data of power consumption, and raw data of error count, forming the first resource monitoring data.

[0068] On each computing node, the node agent program calls the performance monitoring interface provided by the operating system to obtain at least one of the following: central processing unit utilization, memory usage, disk I / O throughput, and network I / O throughput, forming the second resource monitoring data.

[0069] On each computing node, based on the first resource monitoring data and the second resource monitoring data, the raw data of memory usage of each GPU are summarized and statistically analyzed to obtain the used memory capacity, remaining memory capacity and memory fragmentation index of each GPU. The raw data of stream processor utilization and the raw data of dedicated matrix operation unit utilization of each GPU are aggregated by time window to obtain the average computing load and peak computing load of each GPU.

[0070] On each computing node, a health status assessment is performed on each GPU based on raw temperature data, raw power consumption data, and raw error count data to obtain health status information for each GPU. The health status information includes at least one of the following: whether it is in an overheating state, whether it is in a power consumption limit state, and whether there is an error rate anomaly.

[0071] On each computing node, based on the GPU driver interface and topology discovery interface, interconnection information between GPUs and between GPUs and the central processing unit is obtained. The interconnection information includes at least one of interconnection type, interconnection bandwidth and affinity, and the interconnection information is used as the original topology data.

[0072] On each computing node, the used video memory capacity, remaining video memory capacity, video memory fragmentation index, average computing load, peak computing load, health status information, central processing unit utilization, memory usage, disk I / O throughput, network I / O throughput, and topology raw data are encapsulated into a node resource status reporting message according to a preset data format and sent to the cluster management component through the cluster communication channel.

[0073] Finally, at the cluster management component, node resource status reports from each compute node are received. Based on the timestamp information in the messages, the resource status of different compute nodes is time-aligned, and missing sampling points are interpolated to generate a snapshot of resource status information corresponding to each compute node in the GPU cluster. This snapshot of resource status information is then associated with the node identifier of the corresponding compute node and stored for use in subsequent node status construction and determination of the target GPU resource combination.

[0074] Furthermore, Figure 3 This is a flowchart illustrating the node state construction method in S120 according to an example embodiment of this application. For example... Figure 3 As shown, the node state construction method in S120 above includes:

[0075] S121. For each computing node, obtain the memory usage status, computing load status, and health status of each GPU on that computing node.

[0076] Optionally, the above video memory usage status includes: used video memory capacity, remaining video memory capacity, and video memory fragmentation index.

[0077] Optionally, the above computing load status includes: the current utilization of the stream processor and the list of running tasks.

[0078] Optionally, the aforementioned health status includes at least one of the following: temperature information, power consumption information, and error statistics information.

[0079] S122. For each computing node, obtain the topology information between GPUs within that computing node and between the GPUs and the central processing unit.

[0080] Optionally, the topology information includes at least one of the following: the interconnect type between GPUs, the interconnect bandwidth, and the affinity between the GPU and the central processing unit.

[0081] Specifically, on each computing node, the node agent program calls the GPU driver interface and topology discovery interface to obtain the device identification information of each GPU in the computing node and the raw data of the interconnection relationship between each GPU. The raw data of the interconnection relationship includes at least one of the following: link identifier, link type identifier, and link bandwidth identifier, which are used to characterize the physical connection relationship between GPUs.

[0082] Specifically, on each computing node, the node agent program calls the processor topology interface and memory topology interface provided by the operating system to obtain the processor identifier, NUMA node identifier and memory controller identifier of each central processing unit in the computing node, and obtains the distance measurement information between each GPU and the corresponding central processing unit. The distance measurement information includes at least one of NUMA distance, access latency level and available bandwidth level.

[0083] On each computing node, based on the original interconnection data, the interconnection type and interconnection bandwidth between each GPU are determined. The interconnection type includes at least one of PCIe, NVLink, NVSwitch and Ethernet. The interconnection bandwidth is used to characterize the upper limit of available bandwidth and the current occupied bandwidth of the corresponding interconnection link within a preset statistical period.

[0084] On each computing node, the affinity between each GPU and the central processing unit is determined based on the processor identifier, NUMA node identifier, and distance metric information. The affinity includes at least one of the preferred central processing unit, the second-best central processing unit, and the non-preferred central processing unit corresponding to the target GPU.

[0085] On each computing node, using the GPU device identifier as an index, the interconnection types and interconnection bandwidths of other GPUs directly connected to this GPU or connected via a single-hop interconnect are summarized to form local interconnection topology information corresponding to this GPU. The identifiers of central processing units with preferred affinity to this GPU are also associated to form local affinity information corresponding to this GPU.

[0086] On each computing node, the local interconnection topology information, local affinity information, and corresponding GPU device identifier and central processing unit identifier of each GPU are encapsulated into node topology information reporting data according to a preset data format and sent to the cluster management component through the cluster communication channel.

[0087] Finally, at the cluster management component, node topology information reports from each compute node are received. Based on the GPU device identifier and the central processing unit identifier, the topology information reported by different compute nodes is normalized to generate topology information corresponding to each compute node. The topology information includes at least one of the following: interconnection type between GPUs, interconnection bandwidth, and affinity relationship between GPU and central processing unit. This information is used to participate in the generation of node status and the subsequent determination of target GPU resource combinations.

[0088] S123. Generate the node status corresponding to each computing node based on the memory usage status, computing load status, health status, and topology information.

[0089] Specifically, for each computing node, the availability of each GPU's memory is evaluated based on its used memory capacity, remaining memory capacity, and memory fragmentation index. This determines the memory availability level for each GPU, which represents the GPU's ability to meet the memory requirements of new tasks under the current memory usage and fragmentation levels.

[0090] For each compute node, the compute availability of each GPU is evaluated based on the current utilization of the stream processors and the list of running tasks on that compute node. The compute load level corresponding to each GPU is determined, and the compute load level is used to represent the current idle computing power of the GPU and the additional compute load space that can be accommodated.

[0091] For each computing node, the operational reliability of each GPU is evaluated based on at least one of the temperature information, power consumption information, and error statistics information of each GPU on that computing node, in order to determine the health status level corresponding to each GPU. The health status level includes at least one of healthy, restricted, and abnormal, which is used to characterize whether the GPU is suitable as a candidate scheduling resource.

[0092] For each computing node, based on the interconnection type and bandwidth between GPUs within that computing node, as well as the affinity between each GPU and the central processing unit, a topology aggregation analysis is performed on the available GPU combinations within that computing node to determine the topology fit index corresponding to different GPU combinations. The topology fit index is used to characterize the communication efficiency when executing multi-GPU parallel tasks on the corresponding GPU combination.

[0093] For each computing node, the overall resource availability of the computing node is quantified by combining the memory availability level, computing load level and health status level of each GPU on the computing node, so as to generate a first node resource vector corresponding to the computing node. The first node resource vector includes at least one of the following: number of GPUs, total available memory, total available computing power and number of available healthy GPUs.

[0094] For each computing node, a second node topology vector is generated based on the topology adaptability index and the affinity between each GPU and the central processing unit. The second node topology vector includes at least one of the following: the number of high-bandwidth GPU combinations supported within the computing node, the number of low-latency GPU combinations, and the number of GPUs bound to the preferred central processing unit.

[0095] On each computing node, the memory availability level, computing load level, and health status level corresponding to each GPU on that computing node are associated with the resource vector of the first node and the topology vector of the second node to form a node state object corresponding to that computing node. The node state object is used to uniformly describe the comprehensive capabilities of the computing node in terms of memory resources, computing resources, health status, and topology.

[0096] Finally, at the cluster management component, the node status objects reported by each compute node are associated with and stored with the corresponding compute node identifiers to form a node status corresponding to each compute node in the GPU cluster. The node status is used to participate in subsequent resource adaptation weight calculation, affinity calculation, and determination of the target GPU resource combination.

[0097] S130. Determine the scheduling priority of the task to be scheduled based on the task profile and node status.

[0098] Figure 4 This is a flowchart illustrating a specific implementation of S130 according to an example embodiment of this application. Figure 4 As shown, the above-mentioned S130 includes:

[0099] S131. Obtain the static priority of the task to be scheduled based on the task configuration.

[0100] In this step, the task configuration of the task to be scheduled can be parsed to obtain the task type field, business importance field, and service level agreement field corresponding to the task to be scheduled. The task type field is used to represent the task category to which the task to be scheduled belongs, the business importance field is used to represent the relative importance of the business corresponding to the task to be scheduled, and the service level agreement field is used to represent the service level target corresponding to the task to be scheduled.

[0101] Based on the pre-configured task type priority mapping rules, the task type field is mapped to the corresponding task type priority. The task type priority is used to distinguish the basic priority differences of at least one of offline training tasks, online inference tasks, interactive development tasks, and system maintenance tasks.

[0102] Based on pre-configured business importance priority mapping rules, the business importance field is mapped to the corresponding business importance weight, where the business importance weight is used to represent the relative weight of different business lines, different tenants or different projects in resource scheduling.

[0103] Based on the pre-configured service level protocol priority mapping rules, the service level protocol fields are mapped to the corresponding service level weights. The service level weights are used to characterize the differences in task completion time, availability, or throughput requirements among different service level protocols.

[0104] After obtaining the task type priority, business importance weight, and service level weight, the task type priority, business importance weight, and service level weight are weighted and combined to obtain the static priority score corresponding to the task to be scheduled. The static priority score is used to represent the basic scheduling priority order of the task to be scheduled without considering the current cluster running status and waiting time.

[0105] Finally, the static priority score is associated with the task identifier of the task to be scheduled and stored, and the static priority score is written into the task profile for use in subsequent scheduling priority calculations.

[0106] S132. Determine the time weight based on the timeliness requirements of the task to be scheduled and the waiting time.

[0107] In this step, the timeliness requirement field corresponding to the task to be scheduled can be obtained from the task configuration of the task to be scheduled. The timeliness requirement field is used to characterize the intensity of the task to be scheduled's requirements for response latency, completion time limit or throughput. The timeliness requirement field includes at least one of the following: urgent task, real-time task, near real-time task and offline task.

[0108] Based on the pre-configured timeliness requirement level mapping rules, the timeliness requirement field is mapped to the corresponding timeliness level value, where the timeliness level value is used to characterize the basic time sensitivity of different types of timeliness requirements in scheduling.

[0109] Retrieve the waiting time of the scheduled task since it entered the task scheduling queue. The waiting time represents the cumulative queuing time of the scheduled task without GPU resources.

[0110] Based on the pre-configured waiting time segmentation rules, the waiting time is divided into corresponding waiting time intervals, and a corresponding waiting time amplification coefficient is configured for each waiting time interval. The waiting time amplification coefficient is used to gradually increase the time urgency of the scheduled task as the waiting time increases.

[0111] Based on the timeliness level value and the waiting time amplification coefficient, the time weight score corresponding to the task to be scheduled is determined. The time weight is used to mainly reflect the initial time sensitivity represented by the timeliness requirement field when the waiting time is short. When the waiting time exceeds the preset threshold, the weight difference between tasks of different timeliness levels is gradually increased or weight compensation is performed to avoid long-waiting tasks being continuously starved.

[0112] After determining the time weight score, the time weight score is normalized to keep it within a preset range so that it can be combined with static priority, critical path weight, and resource adaptation weight for calculation.

[0113] Finally, the normalized time weight score is associated with the task identifier of the task to be scheduled and stored, and the time weight score is written into the task profile for use in subsequent scheduling priority calculation.

[0114] S133. Determine the critical path weight based on the position of the task to be scheduled in the task dependency graph.

[0115] In this step, task dependency description information corresponding to the task set to which the task to be scheduled belongs can be obtained from the workflow orchestration system or the task submission system. The task dependency description information is used to characterize the pre- and post-dependencies between multiple tasks and to construct a task dependency graph.

[0116] Based on the task dependency description information, the task dependency graph is constructed as a directed acyclic graph structure. Each node in the directed acyclic graph structure represents a specific task, and each directed edge represents the dependency order constraint between the corresponding starting task and the target task.

[0117] Based on the historical runtime information or estimated runtime information of each task in the task dependency graph, the path duration is calculated for at least one path from the start node to the end node in the task dependency graph. The path duration is used to characterize the contribution of the corresponding path to the total task execution completion time.

[0118] Based on the comparison of multiple paths in the task dependency graph according to path duration, one or more paths are identified as critical paths. Critical paths are used to characterize the sequence of tasks that have a decisive impact on the completion time of the overall workflow.

[0119] Then, determine whether the task to be scheduled is located on the critical path. If the task to be scheduled is located on the critical path, mark the task to be scheduled as a critical task, and assign a critical path base weight value higher than that of non-critical tasks based on the pre-configured critical path weight mapping rules.

[0120] When it is determined that the task to be scheduled is located on the critical path, the cumulative estimated runtime of the remaining task links after the task to be scheduled is obtained based on the topological position of the task to be scheduled on the critical path. The cumulative estimated runtime is used to characterize the length of the critical path that the task to be scheduled still needs to complete.

[0121] Based on the pre-configured segmentation rules for the remaining critical path length, the cumulative estimated runtime is mapped to the corresponding remaining path length adjustment coefficient. The remaining path length adjustment coefficient is used to increase the critical path weight when the task to be scheduled is far from the endpoint of the critical path, so as to reduce the potential delay risk of the overall workflow.

[0122] When it is determined that the task to be scheduled is not located on the critical path, the non-critical path influence coefficient is determined based on the degree of indirect influence of the non-critical path on the critical path. The non-critical path influence coefficient is used to appropriately increase the critical path weight when the task to be scheduled is a candidate task that may be promoted to the critical path.

[0123] Then, based on the critical path base weight value, the remaining path length adjustment coefficient, and the non-critical path influence coefficient, the critical path weight score corresponding to the task to be scheduled is determined. The critical path weight score is used to characterize the degree of influence of the task to be scheduled on the overall workflow completion time.

[0124] Finally, the critical path weight score is associated with the task identifier of the task to be scheduled and stored, and the critical path weight score is written into the task profile for use in subsequent scheduling priority calculation.

[0125] S134. Determine the resource adaptation weight based on the resource adaptation of the task profile and node status.

[0126] In this step, the resource characteristics of the task to be scheduled can be determined based on the computing power intensity, memory requirements, and interconnect bandwidth requirements reflected in the task profile. Then, the node resource characteristics are determined based on the available GPU memory, available computing power, and interconnect bandwidth reflected in the node status. Finally, the resource matching weights are determined based on the similarity between the resource characteristics and the node resource characteristics.

[0127] Optionally, the determination of the resource characteristics of the task to be scheduled can be achieved by reading the computational characteristic information corresponding to the task to be scheduled from the task profile. The computational characteristic information includes at least one of the following: average utilization of stream processors, average utilization of dedicated matrix operation units, thread bundle execution efficiency, and computational density characteristics.

[0128] Based on the average utilization of stream processors and the average utilization of dedicated matrix operation units, the computing power utilization ratio of the tasks to be scheduled is determined. Based on the relationship between the computing power utilization ratio and the preset computing power utilization threshold range, the tasks to be scheduled are classified into at least one computing power intensity category among compute-intensive, memory access-intensive, or bandwidth-mixed types.

[0129] Read the memory feature information corresponding to the task to be scheduled from the task profile. The memory feature information includes at least one of the following: peak memory usage, average memory utilization, and memory fragmentation sensitivity index.

[0130] Based on the relationship between peak video memory and preset video memory capacity tier thresholds, the video memory capacity requirement level of the task to be scheduled is determined, and based on the average video memory utilization and video memory fragmentation sensitivity index, the sensitivity level of the task to video memory continuity and video memory fragmentation degree is determined.

[0131] Read the interconnect bandwidth requirement information corresponding to the task to be scheduled from the task profile. The interconnect bandwidth requirement information includes at least one of the following: average interconnect bandwidth utilization rate between GPUs, peak interconnect bandwidth utilization rate between GPUs, and cross-node communication ratio.

[0132] Based on the average and peak utilization rates of the interconnect bandwidth between GPUs, the interconnect bandwidth pressure level of the task to be scheduled is determined, and based on the cross-node communication ratio, the sensitivity of the task to be scheduled to deployment on the same node or across nodes is determined.

[0133] Based on the computing power intensity category, memory capacity requirement level, memory continuity sensitivity level, interconnect bandwidth pressure level, and intra-node deployment sensitivity, a multi-dimensional resource feature vector is constructed for the task to be scheduled according to the preset coding rules. The multi-dimensional resource feature vector is used to characterize the comprehensive resource characteristics of the task to be scheduled in terms of computing power, memory, and interconnect bandwidth.

[0134] Finally, the multidimensional resource feature vector is associated with the task identifier of the task to be scheduled and stored, and the multidimensional resource feature vector is written into the task profile as the resource feature of the task to be scheduled, which is used to participate in the determination of subsequent resource adaptation weights.

[0135] Optionally, the determination of the above node resource characteristics can be achieved by reading the memory usage status corresponding to each GPU on the target computing node from the node status. The memory usage status includes the used memory capacity, the remaining memory capacity, and the memory fragmentation index.

[0136] Then, based on the relationship between the remaining video memory capacity and the preset video memory capacity tier threshold, the available video memory capacity level of the target computing node in the video memory capacity dimension is determined, and based on the relationship between the video memory fragmentation index and the preset fragmentation threshold, the available video memory continuity level of the target computing node in the video memory continuity dimension is determined.

[0137] Read the compute load status corresponding to each GPU on the target compute node from the node status. The compute load status includes the current utilization of the stream processor and the list of running tasks.

[0138] Based on the relationship between the current utilization rate of the stream processor and the preset computing power utilization threshold, the available computing power level of the target computing node in the computing power dimension is determined, and based on the computing density characteristics in the task profile of each running task in the running task list, the potential for releasing available computing power of the target computing node in the computing power dimension is estimated.

[0139] Read the topology information corresponding to the target computing node from the node status. The topology information includes at least one of the following: GPU-to-GPU interconnect type, interconnect bandwidth, and affinity relationship between GPU and central processing unit.

[0140] Based on interconnect type and interconnect bandwidth, and combined with the current interconnect bandwidth usage between GPUs, the available interconnect bandwidth level of the target compute node in the interconnect bandwidth dimension is determined, and based on the affinity between the GPU and the central processing unit, the local interconnect priority of the target compute node in the data locality dimension is determined.

[0141] Then, based on the available video memory capacity level, available video memory continuity level, available computing power level, available computing power release potential, available interconnect bandwidth level, and local interconnect priority, a node resource feature vector is constructed for the target computing node according to the preset encoding rules. The node resource feature vector is used to characterize the comprehensive resource supply capability of the target computing node in the dimensions of video memory, computing power, and interconnect bandwidth.

[0142] Finally, the node resource feature vector is associated with the node identifier of the target computing node and stored, and the node resource feature vector is written into the node state as the node resource feature of the target computing node for subsequent determination of resource adaptation weights.

[0143] Furthermore, after determining the resource characteristics of the task to be scheduled and the node resource characteristics, the determination of the resource adaptation weights can be achieved by representing the resource characteristics determined based on the task profile as a first multi-dimensional feature vector, wherein the first multi-dimensional feature vector includes at least one or more of the dimensions of computing power intensity, video memory requirement, and interconnect bandwidth requirement. The node resource characteristics determined based on the node state can be represented as a second multi-dimensional feature vector, wherein the second multi-dimensional feature vector includes at least one or more of the dimensions of available computing power, available video memory, and available interconnect bandwidth.

[0144] Then, the feature values ​​of each dimension of the first and second multidimensional feature vectors are uniformly normalized so that the feature values ​​of each dimension are mapped to a preset interval.

[0145] Next, based on the computing power intensity information, memory requirement information, and interconnect bandwidth requirement information reflected in the task profile, feature importance coefficients corresponding to the computing power dimension, memory dimension, and interconnect bandwidth dimension are determined. According to the feature importance coefficients, the feature values ​​of the first multidimensional feature vector and the second multidimensional feature vector in each dimension are weighted to obtain the weighted first multidimensional feature vector and the weighted second multidimensional feature vector.

[0146] The initial similarity score between resource features and node resource features is calculated based on the cosine of the angle between the weighted first multidimensional feature vector and the weighted second multidimensional feature vector, or the reciprocal of the Euclidean distance between them.

[0147] Then, the initial similarity score is compared with the preset upper and lower similarity thresholds. When the initial similarity score is higher than the upper similarity threshold, the initial similarity score is truncated to the upper similarity threshold. When the initial similarity score is lower than the lower similarity threshold, the initial similarity score is raised to the lower similarity threshold, so as to obtain the target similarity score after boundary constraint processing.

[0148] Based on the target similarity score and the preset mapping function corresponding to the task type information, the resource adaptation weight is determined to characterize the degree of matching between task resource features and node resource features.

[0149] Finally, the resource adaptation weights are associated with the task identifiers of the corresponding tasks to be scheduled and stored, and the resource adaptation weights are written into the task profile for use in subsequent scheduling priority calculations.

[0150] S135. Based on static priority, time weight, critical path weight, and resource adaptation weight, determine the scheduling priority of the task to be scheduled.

[0151] Specifically, the static priority obtained from the task configuration is normalized to a preset priority range to obtain a static priority score. The time weight, determined based on the timeliness requirements of the task to be scheduled and the waiting time, is normalized to a preset weight range to obtain a time weight score. The critical path weight, determined based on the position of the task to be scheduled in the task dependency graph, is normalized to a preset weight range to obtain a critical path weight score. The resource adaptation weight, determined based on the resource adaptation of the task profile and node status, is normalized to a preset weight range to obtain a resource adaptation weight score.

[0152] Then, based on the priority combination parameters used to characterize scheduling strategy preferences, determine the static priority coefficient, time weight coefficient, critical path weight coefficient, and resource adaptation weight coefficient that correspond one-to-one with the static priority score, time weight score, critical path weight score, and resource adaptation weight score.

[0153] The initial scheduling priority score of the task to be scheduled is determined based on the weighted results of the static priority score and static priority coefficient, the weighted results of the time weight score and time weight coefficient, the weighted results of the critical path weight score and critical path weight coefficient, and the weighted results of the resource adaptation weight score and resource adaptation weight coefficient.

[0154] Then, the initial scheduling priority score is compared with the preset upper and lower limits of scheduling priority. When the initial scheduling priority score is greater than the upper limit of scheduling priority, the initial scheduling priority score is truncated to the upper limit of scheduling priority. When the initial scheduling priority score is less than the lower limit of scheduling priority, the initial scheduling priority score is raised to the lower limit of scheduling priority, so as to obtain the target scheduling priority score after boundary constraint processing.

[0155] Finally, the target scheduling priority score is associated with the task identifier of the corresponding task to be scheduled and stored, and the target scheduling priority score is written into the task profile for use in the subsequent calculation of the candidate GPU resource combination matching score and the determination of the target GPU resource combination.

[0156] S140. Based on the task profile, node status, and scheduling priority, determine the target GPU resource combination for executing the scheduled task.

[0157] Figure 5 This is a flowchart illustrating a specific implementation of S140 according to an example embodiment of this application. For example... Figure 5 As shown, the above-mentioned S140 includes:

[0158] S141. For each candidate GPU resource combination, determine the affinity between the task and the candidate GPU resource combination based on the task profile and the node status corresponding to the candidate GPU resource combination.

[0159] In this step, topology fit can be determined based on the parallelism and interconnect bandwidth requirements in the task profile, as well as the topology information corresponding to candidate GPU resource combinations. Computational power fit can be determined based on the computational and memory characteristics in the task profile, as well as the available computing power and memory availability information corresponding to candidate GPU resource combinations. Locality fit can be determined based on the data locality requirements in the task profile, as well as the node data locations corresponding to candidate GPU resource combinations. Finally, affinity can be determined based on topology fit, computational power fit, and locality fit.

[0160] Optionally, the determination of the topology fit can be achieved by obtaining the target number of GPUs, inter-GPU communication mode, and interconnect bandwidth requirement per unit time for the scheduled task from the task profile, and obtaining the interconnect type, interconnect bandwidth, and distribution of the candidate GPU resource combination on physical nodes from the topology information corresponding to the candidate GPU resource combination. Then, based on the consistency between the target number of GPUs and the number of available GPUs in the same high-speed interconnect domain in the candidate GPU resource combination, a first topology matching score is determined. Furthermore, based on whether the inter-GPU communication mode and the interconnect type and interconnect bandwidth between any two GPUs in the candidate GPU resource combination meet the interconnect bandwidth requirement per unit time, a second topology matching score is determined. Finally, the topology fit is determined based on the weighted result of the first and second topology matching scores.

[0161] Optionally, the determination of the aforementioned computing power suitability can be achieved by obtaining the computational intensity characteristics, stream processor utilization characteristics, peak memory requirements, and average memory utilization characteristics of the task to be scheduled from the task profile, and obtaining the available computing power, current stream processor load, available memory capacity, and memory fragmentation index from the node status corresponding to the candidate GPU resource combinations. Then, based on the degree of matching between the computational intensity characteristics and the available computing power and current stream processor load, a computing power matching score is determined, and based on the degree of matching between the peak memory requirements and average memory utilization characteristics and the available memory capacity and memory fragmentation index, a memory matching score is calculated. Finally, the computing power suitability is determined based on the weighted result of the computing power matching score and the memory matching score.

[0162] Optionally, the determination of the aforementioned locality adaptation can be achieved by obtaining at least one of the following from the task profile: the location of the training dataset, the location of the model parameter storage, and the location of the intermediate result cache for the task to be scheduled. This determines the data locality requirement information of the task to be scheduled. Alternatively, at least one of the following can be obtained from the node status corresponding to the candidate GPU resource combination: the list of locally stored datasets, local model parameter cache information, and network topology and bandwidth information between the node and external storage nodes. This determines the node data location. Then, the locality adaptation is determined based on the weighted result of the data overlap between the data locality requirement information and the node data location, the data access path length, and the cross-node data transmission bandwidth overhead.

[0163] S142. For each candidate GPU resource combination, based on the task profile of the existing running tasks on the candidate GPU resource combination, determine the interference level that will occur when scheduling the task to be scheduled to the candidate GPU resource combination.

[0164] Specifically, the degree of resource conflict can be determined based on the resource usage patterns in the task profile and the resource usage patterns in the task profiles of existing running tasks on candidate GPU resource combinations. Then, the performance impact during concurrent execution can be determined based on the current memory fragmentation index, current stream processor load, and interconnect bandwidth usage in the node status. Finally, the interference level can be determined based on the degree of resource conflict and the degree of performance impact.

[0165] Optionally, the determination of the aforementioned resource conflict level can be achieved by obtaining the memory usage curve, computational load curve, and interconnect bandwidth usage curve at a preset time granularity from the task profile of the task to be scheduled, and by obtaining the corresponding memory usage curve, computational load curve, and interconnect bandwidth usage curve from the task profiles of each existing running task in the candidate GPU resource combination. Then, the curves on the same GPU are aligned according to the time axis, and the overlap ratio of the memory usage interval, the overlap ratio of the computational load interval, and the overlap ratio of the interconnect bandwidth usage interval are calculated in each time slice. Based on the weighted summation of the overlap ratios in each time slice, the resource conflict level between the task to be scheduled and each existing running task is determined.

[0166] Optionally, the determination of the aforementioned performance impact can be achieved by obtaining the current memory fragmentation index, current stream processor utilization, and inter-GPU interconnect bandwidth utilization of each GPU from the node status corresponding to the candidate GPU resource combination; determining the memory allocation penalty score based on the deviation of the current memory fragmentation index from a preset memory fragmentation threshold; then determining the computing power contention penalty score based on the deviation of the current stream processor utilization from a preset utilization threshold, and determining the communication congestion penalty score based on the deviation of the interconnect bandwidth utilization from a preset bandwidth threshold; finally, determining the performance impact during concurrent execution based on the weighted sum of the memory allocation penalty score, computing power contention penalty score, and communication congestion penalty score.

[0167] S143. Based on scheduling priority, affinity, and interference, determine the matching score of candidate GPU resource combinations.

[0168] Specifically, the scheduling priority of the task to be scheduled can be normalized to obtain a priority score, the affinity can be normalized to obtain an affinity score, and the interference can be normalized and inverted to obtain an interference penalty score. Then, based on preset priority weights, affinity weights, and interference weights, the priority score, affinity score, and interference penalty score are weighted and summed to obtain a matching score corresponding to the candidate GPU resource combination.

[0169] S144. Based on the matching score, determine the target GPU resource combination from multiple candidate GPU resource combinations.

[0170] In this step, the matching scores corresponding to each candidate GPU resource combination are sorted to determine the sequence of candidate GPU resource combinations with matching scores from high to low.

[0171] During the process of sequentially traversing the candidate GPU resource combinations, based on the computing power requirement information and video memory requirement information in the task configuration corresponding to the task to be scheduled, it is determined whether the available computing power, available video memory, and interconnect bandwidth of the current candidate GPU resource combination meet the computing power requirement information, video memory requirement information, and preset interconnect bandwidth lower limit constraints.

[0172] If the current candidate GPU resource combination meets the constraints of computing power requirements, memory requirements, and interconnect bandwidth, then the candidate GPU resource combination is determined as the target GPU resource combination.

[0173] In the absence of candidate GPU resource combinations that meet the constraints of computing power requirements, memory requirements, and interconnect bandwidth limits, the candidate GPU resource combination with the highest matching score and a resource gap not exceeding a preset tolerance threshold is selected from multiple candidate GPU resource combinations as the target GPU resource combination.

[0174] In this embodiment, task information of at least one task to be scheduled is obtained, and a corresponding task profile is constructed based on the task information. Resource status information of each computing node in the GPU cluster is obtained, and a corresponding node status is constructed based on the resource status information. Then, the scheduling priority of the task to be scheduled is determined according to the task profile and node status. Then, the target GPU resource combination for executing the task to be scheduled is determined according to the task profile, node status and scheduling priority. Thus, under the conditions of large-scale operation of GPU cluster, diverse task types and dynamic changes in resource requirements, a task profile and node status that can truly reflect the task resource requirements and cluster resource status are constructed before scheduling decisions. On this basis, the matching degree between the scheduling priority of the task to be scheduled and the candidate GPU resource combination is quantified, thereby selecting the target GPU resource combination that can improve the overall resource utilization, reduce inter-task interference and shorten the task completion time while ensuring the timely execution of high-priority tasks.

[0175] In other words, in a GPU cluster, tasks are no longer scheduled simply based on submission order or rough resource requirements. Instead, a task profile containing type, resource requirements, and runtime characteristics is created for each task, and a node status containing memory / computing load, health status, and topology is created for each compute node. Then, the scheduling priority of each task is calculated based on these two types of information, and the best set of GPU resources matching the task is selected from the entire cluster for allocation. This improves the overall GPU utilization and reduces mutual interference between tasks while ensuring that important tasks are executed first.

[0176] Based on the GPU cluster scheduling in the above embodiments, the scheduling decision is completed once when a task is first submitted or first started. Even if the overall load of the GPU cluster changes significantly, the node resource status fluctuates drastically, or higher-priority tasks arrive successively during subsequent operation, the tasks already running will not be systematically re-evaluated and re-planned. Furthermore, the resource usage characteristics of running tasks, and the real-time load and health status of nodes often exhibit dynamic trends. If the scheduling system still performs static scheduling based on the initial task profile and initial node status, it can easily lead to problems such as high-priority tasks experiencing long-term starvation, some GPU resources in the cluster idling or being used inefficiently, and severe resource contention between new and old tasks that cannot be alleviated through rescheduling.

[0177] In other words, the actual resource usage patterns of a task during operation, such as memory usage, computational load, and bandwidth usage, often deviate from the estimates made during the pre-running or configuration phase. As the training phase progresses, the dataset changes, or the algorithm adapts, the task profile itself evolves dynamically. Furthermore, the resource status of GPU cluster nodes, such as available memory, computational load, interconnect bandwidth, and health status, is highly time-sensitive and highly volatile. If only a resource snapshot is taken before task scheduling, there is no mechanism for linked updates and re-evaluation at preset intervals, making it impossible for the scheduler to discover better resource allocation schemes or necessary preemption and migration opportunities in a timely manner.

[0178] In response, Figure 1 Based on the illustrated embodiment, Figure 6 This is a flowchart illustrating a GPU computing resource scheduling method according to another example embodiment of this application. Figure 6 As shown, in this embodiment... Figure 1 Based on the illustrated embodiment, it also includes:

[0179] S210. Reacquire the task profiles of running tasks and tasks to be scheduled at a preset period, and reacquire the resource status information of each computing node in the GPU cluster.

[0180] In this step, the scheduling control component can trigger a task information update instruction according to a preset period. Then, based on the task information update instruction, task configuration parsing and runtime metric collection are performed on running tasks and tasks to be scheduled, respectively, to update the corresponding task configuration, GPU runtime metrics, and I / O runtime metrics. Based on the updated task configuration, GPU runtime metrics, and I / O runtime metrics, an updated task profile is generated to replace the original task profile.

[0181] Then, based on the task information update instruction, the memory usage status, computing load status, health status and topology information of each GPU on each computing node in the GPU cluster are obtained from the cluster monitoring component. Based on the updated memory usage status, computing load status, health status and topology information, the updated node status is generated and the original node status is replaced.

[0182] S220. Based on the updated task profile and the updated node status, recalculate the scheduling priority and corresponding matching score of running tasks and tasks to be scheduled.

[0183] In this step, for each running task and task awaiting scheduling, the time weight is updated based on the timeliness requirement information and waiting time information in the updated task profile. The critical path weight is updated based on the task dependency information in the updated task profile. The resource adaptation weight is updated based on the resource compatibility between the updated task profile and the updated node status. Finally, the scheduling priority of the running task and task awaiting scheduling is recalculated based on the updated time weight, critical path weight, resource adaptation weight, and corresponding static priority.

[0184] Then, for each candidate GPU resource combination, based on the updated task profile and the updated node status corresponding to the candidate GPU resource combination, the affinity between the task and the candidate GPU resource combination is recalculated. Based on the updated task profile of the existing running tasks on the candidate GPU resource combination and the corresponding updated node status, the interference caused by scheduling the task to be scheduled to the candidate GPU resource combination is recalculated. And based on the recalculated scheduling priority, affinity, and interference, the matching score corresponding to the candidate GPU resource combination is recalculated.

[0185] It is worth noting that the specific steps for recalculating the matching score corresponding to the candidate GPU resource combination can be found in [reference needed]. Figure 1 S140 in the illustrated embodiment will not be described again here.

[0186] S230. Based on the recalculated scheduling priority and matching score, determine whether to adjust the scheduling of at least one running task.

[0187] Figure 7 This is a flowchart illustrating a specific implementation of S230 according to an example embodiment of this application. Figure 7 As shown, the above S230 includes:

[0188] S231. If it is determined that a high-priority task cannot obtain the required GPU resources within a preset time limit, at least one low-priority running task that can be preempted shall be identified.

[0189] In this step, the task preemption cost can be determined based on the scheduling priority, task duration, and task checkpoint interval of the running task. Then, a target low-priority running task with a preemption cost not exceeding a preset threshold is selected from multiple low-priority running tasks as the low-priority running task that can be preempted.

[0190] Specifically, the determination of the task preemption cost can be based on the scheduling priority of the running task, determining the task priority penalty coefficient, determining the progress loss time and additional checkpoint overhead time when the task is preempted based on the running task's running duration, estimated remaining duration, and task checkpoint interval information, and then determining the task preemption cost of the running task based on the weighted result of the task priority penalty coefficient, progress loss time, and additional checkpoint overhead time.

[0191] S232. Perform a state saving operation on low-priority running tasks to generate task checkpoints corresponding to the low-priority running tasks.

[0192] In one possible implementation, after receiving a preemption instruction for a low-priority running task, a pause instruction can be sent to the low-priority running task to trigger it to enter a saveable state.

[0193] Based on the task identifier of the low-priority running task, obtain the GPU context information and task running context information corresponding to the low-priority running task. The GPU context information includes at least one of the following: memory data, model parameters, optimizer state, and intermediate activation data. The task running context information includes at least one of the following: program counter, thread state, and key running configuration parameters.

[0194] Then, the GPU context information and task execution context information are written to a preset persistent storage location, and a checkpoint identifier is generated that is associated with the task identifier of the low-priority running task.

[0195] Finally, the checkpoint identifier is registered in the task profile of the low-priority running task so that the low-priority running task can be restored based on the task checkpoint when GPU resources become available.

[0196] However, multi-GPU tasks are essentially distributed programs executed collaboratively by multiple GPU processes. These GPU processes maintain unified model parameters and training progress through collective and peer-to-peer communication, implicitly containing distributed logical clocks such as iteration steps, communication rounds, and pipeline stages. If the scheduler unidirectionally triggers a checkpoint at any given time, and the current iteration stage and communication call position of each GPU process are inconsistent, the following will occur:

[0197] Some GPUs have completed gradient calculations and updated parameters for a certain batch, while others are still in the forward or communication phase of that batch, and some even have incomplete cross-GPU communication operations.

[0198] Saving checkpoints directly in this partially completed intermediate state will cause inconsistencies in the understanding of model state and communication rounds among the GPU processes after recovery, and the communication protocol state machine will not be aligned. This will trigger deadlock phenomena such as collective communication waiting indefinitely and send / recv misalignment, or silent failure with no apparent error but actual incorrect training results.

[0199] In response to S232 above, another possible implementation is to pre-record synchronization point type information and synchronization interval statistics in the task profile of the low-priority running task. When the scheduler determines to perform a state saving operation on the low-priority running task, it sends a checkpoint preparation instruction to each GPU process participating in the execution of the low-priority running task. Each GPU process checks whether it has received the checkpoint preparation instruction at a preset iteration end, collective communication completion, or pipeline stage boundary. If the checkpoint preparation instruction is detected, it waits for the currently initiated cross-GPU communication operation to complete in order to reach a unified synchronization point. When all GPU processes participating in the execution of the low-priority running task have reached the synchronization point and reported the status information of the executable checkpoint, the scheduler issues a checkpoint execution instruction to uniformly perform the state saving operation at the synchronization point.

[0200] Furthermore, regarding the aforementioned unified state saving operation at the synchronization point, specifically, each GPU process participating in the execution of the low-priority task can save its corresponding computational state, communication state, and framework runtime state based on a preset state serialization structure. The computational state includes at least one of model parameters, optimizer state, memory cache information, and random number generator state; the communication state includes at least one of communication round count and GPU process identifiers participating in communication; and the framework runtime state includes at least one of batch count information and data reading progress information. A global version identifier is then assigned to the task checkpoint generated at the synchronization point, and the scheduler records the set of participating GPU processes corresponding to the global version identifier and their corresponding storage locations. Finally, after confirming that all GPU processes participating in the execution of the low-priority task have successfully completed state saving, the task checkpoint corresponding to the global version identifier is marked as a valid checkpoint that can be used for recovery.

[0201] Furthermore, regarding the specific implementation of restoring low-priority running tasks, the target GPU resource combination for restoring low-priority running tasks and the corresponding GPU process mapping relationship can be determined based on the updated node state and the number of participating GPU processes recorded in the task checkpoint. Then, the communication topology corresponding to the low-priority running tasks is initialized on the target GPU resource combination, ensuring that the number of participating GPU processes, their identifier configuration, and the communication world scale match the set of participating GPU processes recorded in the task checkpoint. Next, in each participating GPU process, the corresponding computation state, communication state, and framework running state are loaded based on the global version identifier. After all GPU processes successfully complete state restoration and report restoration completion information to the scheduler, the scheduler issues a continue execution instruction, allowing the low-priority running tasks to continue running in the next execution step after the synchronization point.

[0202] In the above implementation, by explicitly recording the synchronization point type and typical synchronization interval in the task profile, the scheduler no longer rudely interrupts at any time when deciding to preempt. Instead, it drives each GPU process to move forward to the nearest synchronization boundary in its own execution flow through the checkpoint preparation instruction.

[0203] Furthermore, on the GPU process side, checkpoint preparation instructions are only responded to at natural global synchronization points such as the end of an iteration, the completion of collective communication, or the boundary of a pipeline phase. Before entering state saving, all currently initiated cross-GPU communication operations are allowed to complete naturally, ensuring that there are no communication or parameter updates in a half-finished state within the entire distributed system. From a global perspective, this forms a consistent global slice. Performing state saving uniformly on this slice is equivalent to capturing a snapshot of all logically clock-aligned tasks for the multi-GPU tasks. This allows recovery to proceed from this consistent slice by simply re-establishing the GPU process set and communication world configuration consistent with the snapshot. This ensures consistency of execution phases and communication protocols for multi-GPU tasks throughout the entire process from checkpoint to suspension and then to recovery.

[0204] In other words, synchronization point type information and synchronization interval statistics are pre-recorded in the task profile of low-priority running tasks. When the scheduler decides to save the execution state, it sends a checkpoint preparation instruction to each GPU process participating in the execution of the low-priority running task. This ensures that each GPU process only detects the instruction at preset synchronization points such as the end of an iteration, the completion of collective communication, or the boundary of a pipeline stage. After that, it waits for all currently initiated cross-GPU communication operations to be completed before uniformly reporting the status information of the executable checkpoint. The scheduler then issues the checkpoint execution instruction centrally after confirming that all GPU processes have reached the synchronization point. This achieves the following: checkpoint operations only occur at logical synchronization points shared by all GPU processes and without incomplete communication. When restoring based on task checkpoints in the future, each GPU process can continue execution from the same iteration boundary, ensuring that model parameters, optimizer state, batch progress, and communication rounds remain consistent across all GPUs. This reduces the probability of data misalignment or communication deadlock after multi-GPU task recovery, making state saving and recovery under preemptive scheduling stable and usable.

[0205] S233. After the state saving operation is completed, release the GPU resources allocated to low-priority running tasks.

[0206] First, resource release commands can be sent to the GPU driver and cluster resource management components corresponding to low-priority running tasks to trigger the termination or suspension of low-priority running tasks on the corresponding GPU.

[0207] Secondly, based on the task identifier of the low-priority running task, remove the low-priority running task entry from the running task list of the corresponding GPU, and update the running task list in the node status.

[0208] The video memory space and stream processor quota allocated to low-priority running tasks are then reclaimed, and the reclaimed video memory capacity, stream processor utilization, and interconnect bandwidth usage are written into the corresponding resource status information.

[0209] Then, the memory usage, computational load, and interconnect bandwidth usage of the GPUs corresponding to the low-priority running tasks are updated so that the GPU resources are marked as allocable and can participate in the subsequent determination of target GPU resource combinations.

[0210] S234. Allocate GPU resources to high-priority tasks and perform scheduling based on the task profile and corresponding node status of the high-priority tasks.

[0211] Optionally, the GPU resources released by low-priority running tasks can be matched with the resource characteristics in the task profile of high-priority tasks to determine the target GPU resource combination that meets the memory requirements, computing power requirements, and interconnect bandwidth requirements of high-priority tasks.

[0212] Then, the target GPU resources are combined and written into the task profile corresponding to the high-priority task, and the task identifier of the high-priority task and its allocated GPU resource information are registered in the corresponding node status, and the running task list and resource usage status are updated.

[0213] Then, a task start instruction is sent to the GPU driver and cluster resource management component corresponding to the target GPU resource combination. The task initialization is completed based on the task configuration and checkpoint information in the task profile of the high-priority task, and the execution of the high-priority task is started on the target GPU resource combination.

[0214] S235. When GPU resources are restored to availability, resume low-priority running tasks based on task checkpoints and updated node states.

[0215] Specifically, when it is detected that there are available GPU resources in the GPU cluster that meet the memory, computing power, and interconnect bandwidth requirements of low-priority running tasks, the target GPU resource combination can be determined for the low-priority running tasks based on the updated node status.

[0216] Then, based on the task checkpoint identifier in the task profile of the low-priority running task, the task checkpoint data corresponding to the task checkpoint identifier is loaded, and the task context reconstruction and memory data recovery are completed on the computing node corresponding to the target GPU resource combination.

[0217] After completing the task context reconstruction and memory data recovery, the running status of the low-priority running task is updated from the paused state to the running state, and the task identifier of the low-priority running task and its allocated GPU resource information are registered in the corresponding node status so that the low-priority running task can continue to be executed on the target GPU resource combination.

[0218] exist Figure 6 In the illustrated embodiment, the task profile and node status are configured as a periodically updatable dynamic view, and configured to perform iterative optimization based on the latest view. Specifically, the scheduling system collects the latest running characteristics of running tasks and tasks to be scheduled according to a preset period, updates their resource requirements, resource occupancy patterns, and priority-related weights in the task profile, and simultaneously pulls real-time status indicators such as video memory, computing power, bandwidth, and health from each computing node to form a new node status snapshot. Based on this, the system re-executes the corresponding priority calculation and matching score calculation process, performs incremental or full re-evaluation of all task-resource combinations, and determines whether the running tasks need to be adjusted based on the updated score results, thereby dynamically approximating the optimal or near-optimal state of task priority and resource matching at the current moment.

[0219] This allows the scheduler to promptly identify and trigger rescheduling or preemption operations when high-priority task resources are scarce, node load is significantly unbalanced, or the matching degree between certain running tasks and the current resource combination is significantly reduced. This shortens the waiting time of high-priority tasks, improves the overall utilization of GPU resources and the stability of task completion latency, and achieves a global scheduling quality improvement for the cluster in dynamic environments.

[0220] Furthermore, in actual GPU cluster scheduling scenarios, the performance scaling relationship between tasks and the number of GPUs is highly heterogeneous. For different algorithms, model sizes, and communication modes, the throughput increase from increasing the number of GPUs is not linear; in fact, after exceeding a certain number of GPUs, the marginal benefit rapidly declines or even becomes negative. In addition, during task execution, the parallelism gains of tasks and the cluster load status are typically not reassessed. This results in existing tasks not being able to utilize the extra GPUs to improve throughput when the cluster transitions from busy to idle, and conversely, when the cluster transitions from idle to busy, some tasks continuously occupy a large number of GPUs, causing prolonged starvation of other tasks, thus exacerbating resource waste and uneven waiting times.

[0221] Especially in multi-tenant clusters and scenarios with large load fluctuations, if when a task is submitted, the cluster is limited to finding and allocating available resource combinations in units of a fixed number of GPUs, it is more likely to result in some tasks occupying too many GPU resources for a long time without achieving linear acceleration, while a large number of tasks waiting to be scheduled are queued up due to a lack of enough idle GPUs in a short period of time, making it difficult for the overall GPU utilization and task completion efficiency of the cluster to reach the optimal level.

[0222] Therefore, based on the above embodiments, Figure 8 This is a flowchart illustrating a GPU computing resource scheduling method according to yet another example embodiment of this application. Figure 8 As shown, this embodiment, based on the above embodiments, also includes:

[0223] S310. Construct parallelism capability information for at least one task to be scheduled. The parallelism capability information is used to represent the performance variation relationship under different numbers of GPUs.

[0224] In this step, during the pre-run analysis phase, multiple pre-run experiments are performed on the task to be scheduled under different GPU configurations. The throughput and single-step execution time of each pre-run experiment are then collected. Based on the throughput and single-step execution time of each pre-run experiment, the performance indicators corresponding to different GPU numbers are determined. Finally, the performance indicators are correlated with the corresponding GPU number to generate parallelism capability information.

[0225] Optionally, during the task pre-run analysis phase, under the condition of configuring different numbers of GPUs such as 1, 2, and 4, multiple sets of pre-run experiments are performed on the task to be scheduled, and the performance indicators such as throughput and single-step execution time of each set of experiments are collected. These performance indicators are then correlated with the corresponding number of GPUs to form parallelism capability information representing the performance change relationship under different numbers of GPUs.

[0226] S320. Based on the current GPU cluster load and parallelism capabilities, determine the initial number of GPUs for the tasks to be scheduled.

[0227] During scheduling, the scheduler combines the parallelism capability information with the overall load of the current GPU cluster, the number of idle GPUs, and the status of the waiting queue. Through a preset objective function, such as maximizing the throughput per unit GPU or weighted optimization of the overall task completion time, the scheduler calculates an initial number of GPUs for the task, so that the task can achieve a better trade-off between performance and resource efficiency in the current resource environment.

[0228] S330. During the execution of the task to be scheduled, the number of GPUs allocated to the task to be scheduled is dynamically adjusted based on the GPU cluster load changes and parallelism capability information.

[0229] During task execution, the scheduler periodically senses changes in cluster load and task running status. Combining parallelism capability information, it reassesses the impact of further increasing or decreasing the number of GPUs on task performance and overall cluster throughput. When the marginal benefit of increasing GPU allocation is high and system resources are sufficient, the number of GPUs for the task is dynamically expanded. When reducing GPU allocation has a limited impact on task performance and cluster resources are tight, the task is scaled down. The underlying resource management components realize online allocation of GPUs and corresponding adjustments to the task communication topology, thereby completing closed-loop dynamic optimization control of the number of GPUs for the task.

[0230] use Figure 7 The illustrated embodiment automatically allocates a calculated compromise number of GPUs to more tasks when resources are scarce. This increases the number of tasks started and completed per unit time without significantly sacrificing single-task performance, reducing task queuing time. When resources are abundant, the number of GPUs is automatically increased for tasks with good parallel scalability, improving their throughput and shortening completion time. Simultaneously, tasks with poor parallel scalability avoid blindly increasing the number of GPUs, reducing unnecessary resource consumption. This improves GPU utilization and throughput at the entire cluster level, achieving dual optimization of task performance and resource utilization.

[0231] Furthermore, GPU clusters typically deploy multiple GPU models simultaneously, such as general-purpose GPUs, high-memory GPUs, high-computing-power training GPUs, and inference-optimized GPUs. However, mainstream scheduling systems often only consider GPU availability, sufficient memory, and the number of GPUs as primary constraints when scheduling tasks. They do not differentiate between the capabilities of different GPU models in terms of computing power, memory, bandwidth, and interconnectivity, nor do they have a detailed model of the sensitivity of specific tasks to these capability dimensions. This results in a large number of lightweight tasks being assigned to high-end GPUs without fully utilizing their computing power, or memory-sensitive tasks being assigned to GPU models with smaller memory, frequently triggering memory paging, OutOfMemoryError (OOM), or severe performance degradation. Consequently, heterogeneous GPU resources cannot be efficiently utilized, and the overall cluster cost-effectiveness and task execution efficiency are significantly affected.

[0232] In this regard, based on the above embodiments, Figure 9 This is a flowchart illustrating a GPU computing resource scheduling method according to yet another example embodiment of this application. For example... Figure 9 As shown, this embodiment, based on the above embodiments, also includes:

[0233] S410: Build a GPU capability vector for each GPU model in the GPU cluster. The GPU capability vector includes at least one of the following: computing power capability parameters, memory capability parameters, bandwidth capability parameters, and interconnect capability parameters.

[0234] Specifically, for each GPU model, at least one of the following parameters can be obtained: computing power, memory capacity, bandwidth, and interconnect capability. These parameter values ​​are then normalized and mapped to obtain a corresponding multi-dimensional numerical vector. This multi-dimensional numerical vector is then associated with and stored as the GPU model identifier, serving as the GPU capability vector corresponding to that GPU model.

[0235] S420. Construct a capability sensitivity vector for each task to be scheduled in the task profile. The capability sensitivity vector is used to represent the sensitivity of the task to different capability parameters.

[0236] Specifically, based on the task type information, computing power requirement information, video memory requirement information, and bandwidth requirement information recorded in the task configuration of the task to be scheduled, the influence weight of each parameter in the computing power capability parameter, video memory capability parameter, bandwidth capability parameter, and interconnect capability parameter can be determined.

[0237] The weights of each influence are normalized to obtain the corresponding capability sensitivity scores. Then, the capability sensitivity scores are combined into a multi-dimensional numerical vector according to a preset dimension order, and the multi-dimensional numerical vector is written into the task profile as the capability sensitivity vector corresponding to the task to be scheduled.

[0238] S430 calculates the suitability of the scheduled task for different GPU models based on GPU capability vector and capability sensitivity vector.

[0239] In this step, for each GPU model, the GPU capability vector corresponding to that GPU model and the capability sensitivity vector corresponding to the task to be scheduled are obtained. Then, based on at least one similarity metric method among the weighted inner product of the GPU capability vector and the capability sensitivity vector, cosine similarity, or Euclidean distance, the fit score between the task to be scheduled and the GPU model is calculated. Finally, the fit score is associated with the corresponding GPU model identifier and stored as the fit of the task to be scheduled to different GPU models.

[0240] S440. Based on the compatibility, prioritize scheduling the tasks to be scheduled to the GPU resource combination corresponding to the GPU model with higher compatibility.

[0241] Specifically, for each candidate GPU resource combination, the corresponding adaptation score is obtained based on the GPU models and their quantities contained in the candidate GPU resource combination, and the model adaptation score corresponding to the candidate GPU resource combination is determined based on the weighted average of the adaptation scores.

[0242] Then, the model compatibility score is used as a component of the affinity or matching score of candidate GPU resource combinations, and... Figure 1 In the embodiment shown, the scheduling priority, affinity, and interference of S140 are weighted and fused.

[0243] The candidate GPU resource combination with the highest model adaptation score and the best overall matching score is selected from multiple candidate GPU resource combinations and used as the target GPU resource combination for executing the scheduled task.

[0244] exist Figure 9 In the illustrated embodiment, during cluster initialization or hardware changes, the theoretical and measured core capability parameters of each GPU model are extracted, including but not limited to single-precision / half-precision peak computing power, tensor core computing power, memory capacity and memory bandwidth, PCIe or NVLink bandwidth, typical latency indicators, etc., and these parameters are normalized to form a GPU capability vector.

[0245] Meanwhile, during task pre-run analysis and long-term operation, based on information such as computational intensity characteristics, memory usage curves, I / O and communication characteristics in the task profile, the task's sensitive weights in dimensions such as computing power, memory, bandwidth and interconnection are learned or configured to form a capability sensitivity vector.

[0246] During scheduling decisions, the task's capability sensitivity vector is calculated by using vector dot product, weighted similarity, or other distance metrics to obtain the task's suitability score for different GPU models. Then, combined with the available number and load of each GPU model, a two-level matching process is constructed, from task to GPU model and then to specific GPU instance. This allows the scheduler to prioritize GPU models with higher suitability when generating candidate GPU resource combinations.

[0247] Furthermore, on the one hand, computationally intensive tasks are prioritized on GPU models with strong computing power and high interconnectivity, memory-sensitive or large-model tasks are prioritized on GPU models with high memory bandwidth and large memory capacity, and bandwidth or communication-sensitive tasks are prioritized on nodes with excellent interconnectivity, thereby reducing performance bottlenecks and insufficient memory / bandwidth problems caused by GPU model mismatch.

[0248] On the other hand, it automatically directs lightweight tasks that do not require high computing power, video memory, or bandwidth to lower-cost GPU models, avoiding the use of high-end GPUs and increasing the proportion of high-end GPUs being fully utilized by suitable tasks. Thus, without increasing hardware costs, it simultaneously improves the performance of single tasks and the resource utilization of the global GPU cluster, thereby solving the problems of low resource utilization efficiency and serious performance waste of heterogeneous GPUs.

[0249] Figure 10 This is a schematic diagram illustrating the structure of a GPU computing resource scheduling system according to an example embodiment of this application. Figure 10 As shown, the GPU computing resource scheduling system 500 provided in this embodiment includes:

[0250] The task information acquisition module 510 is used to acquire task information of at least one task to be scheduled, and to construct a corresponding task profile based on the task information.

[0251] The status information acquisition module 520 is used to acquire the resource status information of each computing node in the GPU cluster, and construct the corresponding node status based on the resource status information.

[0252] The priority determination module 530 is used to determine the scheduling priority of the task to be scheduled based on the task profile and the node status.

[0253] The resource scheduling module 540 is used to determine the target GPU resource combination for executing the scheduled task based on the task profile, the node status, and the scheduling priority.

[0254] Figure 11 This is a schematic diagram of the structure of an electronic device according to an example embodiment of this application. For example... Figure 11 As shown, the electronic device 600 provided in this embodiment includes: a processor 601 and a memory 602; wherein:

[0255] Memory 602 is used to store computer programs, and the memory may also be flash memory.

[0256] Processor 601 is used to execute the execution instructions stored in the memory to implement the various steps in the above method. For details, please refer to the relevant descriptions in the preceding method embodiments.

[0257] Alternatively, the memory 602 can be either standalone or integrated with the processor 601.

[0258] When the memory 602 is a device independent of the processor 601, the electronic device 600 may further include:

[0259] Bus 603 is used to connect the memory 602 and the processor 601.

[0260] This embodiment also provides a readable storage medium storing a computer program, which, when executed by at least one processor of an electronic device, enables the electronic device to perform the methods provided in the various embodiments described above.

[0261] This embodiment also provides a program product including a computer program stored in a readable storage medium. At least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program to cause the electronic device to perform the methods provided in the various embodiments described above.

[0262] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the claims.

[0263] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A GPU computing resource scheduling method, characterized in that, include: Obtain task information for at least one task to be scheduled, and construct a corresponding task profile based on the task information; including: Parse the task configuration of the task to be scheduled to obtain task type information, computing power requirement information and timeliness requirement information; Perform pre-run analysis on the task to be scheduled, and collect GPU performance metrics and I / O performance metrics; Based on the task configuration, the GPU performance metrics, and the I / O performance metrics, a task profile corresponding to the task to be scheduled is generated; Obtain the resource status information of each computing node in the GPU cluster, and construct the corresponding node status based on the resource status information, wherein the node status includes a memory fragmentation index that reflects the degree of memory fragmentation. Based on the task profile and the node status, determine the scheduling priority of the task to be scheduled; Based on the task profile, the node status, and the scheduling priority, determine the target GPU resource combination for executing the scheduled task; including: For each candidate GPU resource combination, the affinity between the task and the candidate GPU resource combination is determined based on the task profile and the node status corresponding to the candidate GPU resource combination. For each candidate GPU resource combination, based on the task profile of the existing running tasks on the candidate GPU resource combination, determine the interference level that would occur if the task to be scheduled were scheduled to the candidate GPU resource combination. Based on the scheduling priority, affinity, and interference, a matching score is determined for the candidate GPU resource combination. The target GPU resource combination is determined from multiple candidate GPU resource combinations based on the matching score.

2. The GPU computing resource scheduling method according to claim 1, characterized in that, The pre-run analysis of the task to be scheduled includes: Allocate pre-running GPU resources for the task to be scheduled from the preset GPU resource pool; During the pre-running phase of the scheduled tasks within a preset time period, information on video memory usage and computing load is collected. Based on the memory usage information and the computing load information, the peak memory utilization, average utilization, and computing intensity characteristics of the task to be scheduled are determined. The peak memory usage, average utilization, and computational intensity characteristics are written into the task profile.

3. The GPU computing resource scheduling method according to claim 1 or 2, characterized in that, The GPU performance metrics include at least one of the following: memory usage curve, stream processor utilization, dedicated matrix operation unit utilization, thread bundle execution efficiency, instruction type distribution, global memory bandwidth utilization, cache hit rate, and GPU interconnect bandwidth utilization.

4. The GPU computing resource scheduling method according to claim 1 or 2, characterized in that, The I / O performance metrics include at least one of disk I / O throughput and network I / O throughput.

5. The GPU computing resource scheduling method according to claim 1, characterized in that, The construction of the corresponding node status based on the resource status information includes: For each compute node, obtain the memory usage status, compute load status, and health status of each GPU on that compute node; For each computing node, obtain the topology information between GPUs within that computing node and between GPUs and the central processing unit; Based on the memory usage status, computing load status, health status, and topology information, a node status corresponding to each computing node is generated.

6. The GPU computing resource scheduling method according to claim 5, characterized in that, The computing load status includes: the current utilization of the stream processor and the list of running tasks.

7. The GPU computing resource scheduling method according to claim 5, characterized in that, The health status includes at least one of the following: temperature information, power consumption information, and error statistics information.

8. The GPU computing resource scheduling method according to claim 5, characterized in that, The topology information includes at least one of the following: the interconnect type between GPUs, the interconnect bandwidth, and the affinity between the GPU and the central processing unit.

9. The GPU computing resource scheduling method according to claim 1, characterized in that, The step of determining the scheduling priority of the task to be scheduled based on the task profile and the node status includes: Obtain the static priority of the task to be scheduled based on the task configuration; The time weight is determined based on the timeliness requirements of the task to be scheduled and the waiting time. Determine the critical path weight based on the position of the task to be scheduled in the task dependency graph; Based on the resource adaptation between the task profile and the node status, determine the resource adaptation weight; The scheduling priority of the task to be scheduled is determined based on the static priority, the time weight, the critical path weight, and the resource adaptation weight.

10. The GPU computing resource scheduling method according to claim 9, characterized in that, The step of determining the resource adaptation weight based on the resource adaptation between the task profile and the node status includes: Based on the computing power intensity information, video memory demand information, and interconnect bandwidth demand information reflected in the task profile, the resource characteristics of the task to be scheduled are determined. Based on the available GPU memory, available computing power, and interconnect bandwidth reflected in the node status, the node resource characteristics are determined. The resource adaptation weight is determined based on the similarity between the resource features and the node resource features.

11. The GPU computing resource scheduling method according to claim 1, characterized in that, The determination of the affinity between the task and the candidate GPU resource combination includes: Based on the parallelism requirement information and interconnect bandwidth requirement information in the task profile, as well as the topology information corresponding to the candidate GPU resource combination, the topology fit is determined. Based on the computational and memory features in the task profile, as well as the available computing power and memory availability information corresponding to the candidate GPU resource combinations, the computing power fit is determined. Based on the data locality requirement information in the task profile and the node data location corresponding to the candidate GPU resource combination, the locality adaptation is determined. The affinity is determined based on the topology adaptability, the computing power adaptability, and the locality adaptability.

12. The GPU computing resource scheduling method according to claim 1, characterized in that, The determination of the interference generated when scheduling the task to be scheduled to the candidate GPU resource combination includes: The degree of resource conflict is determined based on the resource usage patterns in the task profiles and the resource usage patterns in the task profiles of existing running tasks on the candidate GPU resource combinations. The performance impact during concurrent execution is determined based on the current memory fragmentation index, current stream processor load, and interconnect bandwidth usage in the node status. The interference level is determined based on the degree of resource conflict and the degree of performance impact.

13. The GPU computing resource scheduling method according to claim 1, characterized in that, Also includes: The task profiles of running tasks and tasks to be scheduled are reacquired at preset intervals, and the resource status information of each computing node in the GPU cluster is reacquired. Based on the updated task profile and the updated node status, the scheduling priorities and corresponding matching scores of running tasks and tasks to be scheduled are recalculated. Based on the recalculated scheduling priority and matching score, determine whether to adjust the scheduling of at least one running task.

14. The GPU computing resource scheduling method according to claim 13, characterized in that, The scheduling and adjustment of at least one running task includes: If it is determined that a high-priority task cannot acquire the required GPU resources within a preset time limit, at least one low-priority task that can be preempted shall be identified. Perform a state saving operation on the low-priority running tasks to generate task checkpoints corresponding to the low-priority running tasks; After the state saving operation is completed, the GPU resources allocated to the low-priority running task are released; The GPU resources are allocated to the high-priority tasks, and scheduling is performed based on the task profiles and corresponding node states of the high-priority tasks. When GPU resources become available again, the low-priority running tasks are resumed based on the task checkpoints and the updated node status.

15. The GPU computing resource scheduling method according to claim 14, characterized in that, The determination of at least one low-priority running task that can be preempted includes: Based on the scheduling priority, task duration information, and task checkpoint interval information of running tasks, determine the task preemption cost; Select a target low-priority running task whose task preemption cost is no higher than a preset threshold from a plurality of low-priority running tasks, and use it as the low-priority running task that can be preempted.

16. The GPU computing resource scheduling method according to claim 1, characterized in that, Also includes: Parallelism capability information is constructed for at least one task to be scheduled, and the parallelism capability information is used to represent the performance variation relationship under different numbers of GPUs; Based on the current GPU cluster load and the parallelism capability information, determine the initial number of GPUs for the tasks to be scheduled; During the execution of the task to be scheduled, the number of GPUs allocated to the task to be scheduled is dynamically adjusted based on the changes in GPU cluster load and the parallelism capability information.

17. The GPU computing resource scheduling method according to claim 16, characterized in that, The process of constructing parallelism capability information for at least one task to be scheduled includes: During the pre-run analysis phase, multiple pre-run experiments were performed on the tasks to be scheduled under different GPU quantity configurations. Collect the throughput and single-step execution time of each pre-run experiment; Based on the throughput and single-step execution time of each pre-run experiment, the performance metrics corresponding to different numbers of GPUs are determined; The performance metrics are correlated with the corresponding number of GPUs to generate the parallelism capability information.

18. The GPU computing resource scheduling method according to claim 1, characterized in that, Also includes: A GPU capability vector is constructed for each GPU model in the GPU cluster. The GPU capability vector includes at least one of the following: computing power capability parameters, memory capability parameters, bandwidth capability parameters, and interconnect capability parameters. In the task profile, a capability sensitivity vector is constructed for each task to be scheduled. The capability sensitivity vector is used to represent the sensitivity of the task to different capability parameters. Based on the GPU capability vector and the capability sensitivity vector, calculate the adaptability of the task to be scheduled to different GPU models; Based on the compatibility, the tasks to be scheduled will be preferentially scheduled to the GPU resource combination corresponding to the GPU model with higher compatibility.

19. A GPU computing resource scheduling system, characterized in that, include: The task information acquisition module is used to acquire task information of at least one task to be scheduled, and to construct a corresponding task profile based on the task information. The task information acquisition module is specifically used for: Parse the task configuration of the task to be scheduled to obtain task type information, computing power requirement information and timeliness requirement information; Perform pre-run analysis on the task to be scheduled, and collect GPU performance metrics and I / O performance metrics; Based on the task configuration, the GPU performance metrics, and the I / O performance metrics, a task profile corresponding to the task to be scheduled is generated; The status information acquisition module is used to acquire the resource status information of each computing node in the GPU cluster, and construct the corresponding node status based on the resource status information. The node status includes a memory fragmentation index that reflects the degree of memory fragmentation. The priority determination module is used to determine the scheduling priority of the task to be scheduled based on the task profile and the node status. The resource scheduling module is used to determine the target GPU resource combination for executing the scheduled task based on the task profile, the node status, and the scheduling priority. The resource scheduling module is specifically used for: For each candidate GPU resource combination, the affinity between the task and the candidate GPU resource combination is determined based on the task profile and the node status corresponding to the candidate GPU resource combination. For each candidate GPU resource combination, based on the task profile of the existing running tasks on the candidate GPU resource combination, determine the interference level that would occur if the task to be scheduled were scheduled to the candidate GPU resource combination. Based on the scheduling priority, affinity, and interference, a matching score is determined for the candidate GPU resource combination. The target GPU resource combination is determined from multiple candidate GPU resource combinations based on the matching score.

20. An electronic device, characterized in that, include: processor; as well as, Memory for storing the executable instructions of the processor; The processor is configured to execute the method of any one of claims 1 to 18 by executing the executable instructions.

21. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 18.

Images