Multi-tenant dynamic scheduling method and system, device for heterogeneous computing tasks

Abstract

The present application relates to a kind of multi-tenant dynamic scheduling method and system for heterogeneous computing task and computer equipment.A kind of multi-tenant dynamic scheduling method for heterogeneous computing task of the present application includes: obtaining the multidimensional index data of work node in distributed computing cluster, according to task demand and multidimensional index data, multiple candidate work nodes are filtered out;According to multidimensional index data, the multidimensional resource portrait of each candidate work node is calculated, and candidate work node combination is generated according to multidimensional resource portrait;According to multidimensional tenant attribute, the comprehensive scheduling weight of the tenant to which heterogeneous computing task belongs is calculated;The corresponding topology score of each candidate work node combination is calculated, and the comprehensive scheduling weight and the card type matching degree of task demand and candidate work node combination are combined, to obtain comprehensive scheduling score;According to comprehensive scheduling score, target work node combination is determined from candidate work node combination, and heterogeneous computing task is scheduled to target work node combination and executes.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence / cloud computing technology, specifically to a multi-tenant dynamic scheduling method, system, and device for heterogeneous computing tasks. Background Technology

[0002] In existing distributed AI training architectures, a typical training task usually consists of multiple worker nodes, which are often scheduled to run in parallel on different physical nodes in the cluster. During training, the worker nodes need to frequently perform gradient synchronization communication (such as All-Reduce operations) to update model parameters, and the communication efficiency directly affects the training speed.

[0003] A Kubernetes (K8s) cluster is essentially a distributed operating system, with worker nodes serving as the underlying computing servers supporting its operation. In real-world business scenarios within intelligent computing centers, multiple tenants (such as different R&D teams or business lines) often need to simultaneously run large-scale distributed AI training tasks. These training tasks consume significant computing power; therefore, these tenants typically need to share heterogeneous GPU resources (including different types of GPU cards, such as high-performance cards, mid-range cards, and domestically produced accelerator cards) across the worker nodes in the cluster. Different tenants have different SLA levels (such as high priority, medium priority, and low priority) and varying requirements for resource guarantees and scheduling priorities. Consequently, existing Kubernetes cluster scheduling mechanisms have significant limitations when facing complex heterogeneous multi-tenant scenarios. Summary of the Invention

[0004] This invention aims to solve at least one of the technical problems existing in the prior art, and proposes a multi-tenant dynamic scheduling method, system, and device for heterogeneous computing tasks.

[0005] To achieve the above objectives, this invention provides a multi-tenant dynamic scheduling method for heterogeneous computing tasks, applied to a distributed computing cluster, wherein the distributed computing cluster includes multiple worker nodes, characterized in that it includes:

[0006] Obtain heterogeneous computing tasks, parse the heterogeneous computing tasks to obtain task requirements, and determine the tenant to which the heterogeneous computing tasks belong and the multi-dimensional tenant attributes corresponding to the tenant.

[0007] Obtain multi-dimensional indicator data of the worker nodes in the distributed computing cluster, and filter out multiple candidate worker nodes based on the task requirements and the multi-dimensional indicator data of the worker nodes.

[0008] Calculate a multidimensional resource profile for each candidate working node based on the multidimensional indicator data of the candidate working nodes, and generate at least one combination of candidate working nodes based on the multidimensional resource profiles.

[0009] The comprehensive scheduling weight of the tenant to which the heterogeneous computing task belongs is calculated based on the multidimensional tenant attributes;

[0010] For each candidate work node combination, calculate the corresponding topology score, and combine it with the comprehensive scheduling weight and the card type matching degree between the task requirements and the candidate work node combination to obtain the comprehensive scheduling score;

[0011] Based on the comprehensive scheduling score, a target working node combination is determined from the at least one group of candidate working node combinations, and the heterogeneous computing task is scheduled to be executed in the target working node combination.

[0012] In some embodiments, the working node is configured with a heterogeneous acceleration card;

[0013] The heterogeneous accelerator card includes at least one of a graphics processing unit, a neural network processor, and a field-programmable gate array;

[0014] The multidimensional indicator data includes at least one of the following: accelerator card model, video memory capacity, node internal interconnection link status, inter-node communication network topology, accelerator card computing unit utilization rate, and allocated video memory.

[0015] In some embodiments, the multi-dimensional resource profile includes: card performance vector, memory fragmentation rate, topology matrix, and current computing power utilization.

[0016] The calculation of the multidimensional resource profile based on the multidimensional indicator data specifically includes:

[0017] Determine the card type performance vector based on the accelerator card model;

[0018] The video memory fragmentation rate is calculated based on the video memory capacity and the amount of video memory already allocated.

[0019] Construct a topology matrix based on the internal interconnection link status of the nodes;

[0020] The current computing power utilization rate is evaluated based on the occupancy rate of the accelerator card's computing units.

[0021] In some embodiments, the multidimensional tenant attributes include at least one of SLA level, resource quota limit, historical usage efficiency statistics, and task urgency label;

[0022] The comprehensive scheduling weight of the tenant to which the heterogeneous computing task belongs is calculated based on the multidimensional tenant attributes, including: the comprehensive scheduling weight is obtained by weighting the SLA level, the historical usage efficiency statistics and the task urgency label.

[0023] In some embodiments, generating at least one set of candidate working node combinations based on the multidimensional resource profile specifically includes:

[0024] Based on the multi-dimensional resource profile, candidate working nodes that meet preset conditions are selected from the multiple working nodes;

[0025] Based on the group scheduling strategy, a node combination that meets the requirements of the number of working nodes and card type for the heterogeneous computing task is selected from the candidate working nodes.

[0026] Obtain the physical communication link status between each working node in the distributed computing cluster, parse the cluster communication topology graph, and calculate the topology connectivity score within each node combination based on the cluster communication topology graph.

[0027] The node combinations whose topological connectivity scores meet a preset threshold are determined as the candidate working node combinations.

[0028] In some embodiments, before scheduling the heterogeneous computing task to the target worker node combination, the method further includes: attempting to atomically bind all worker nodes in the target worker node combination; if the binding of any worker node in the target worker node combination fails, performing a rollback operation on the bound worker nodes and regenerating the target worker node combination until all worker nodes in the target worker node combination are successfully bound.

[0029] In some embodiments, after all worker nodes in the target worker node combination have been successfully bound, the method further includes:

[0030] The task profile of the heterogeneous computing task is analyzed, and the task profile includes GPU utilization estimation, memory requirements and memory isolation identifier.

[0031] Based on the task profile, the target GPU partitioning mode is determined from the dedicated GPU mode, the multi-instance GPU space-sharing mode, and the virtual GPU time-sharing mode.

[0032] Allocate GPU resources to the heterogeneous computing task according to the target GPU partitioning mode, and start the heterogeneous computing task.

[0033] In some embodiments, based on the task profile, the target GPU partitioning mode is determined from the following: dedicated GPU mode, multi-instance GPU space-sharing mode, and virtual GPU time-sharing mode, including:

[0034] When the estimated GPU utilization rate is higher than the preset utilization rate threshold, or when the memory isolation flag indicates that strong isolation is required, the heterogeneous computing task is allocated an exclusive GPU mode or a MIG space division mode.

[0035] When the estimated GPU utilization is lower than the preset utilization threshold and the memory isolation flag indicates that sharing is allowed, vGPU time-sharing mode is allocated to the heterogeneous computing task.

[0036] In some embodiments, allocating GPU resources to the heterogeneous computing task according to the target GPU partitioning mode includes:

[0037] The node device plugin reports to the distributed computing cluster the allocable GPU resource instances and quantities exposed by each worker node in the target worker node combination under the target GPU partitioning mode.

[0038] On the same worker node in the target worker node combination, MIG space-division mode and vGPU time-division sharing mode are run respectively to support the hybrid deployment of heterogeneous GPU resources.

[0039] This invention also provides a multi-tenant dynamic scheduling system for heterogeneous computing tasks, applied to a distributed computing cluster, wherein the distributed computing cluster includes multiple worker nodes, and the system includes:

[0040] The task analysis module is used to acquire heterogeneous computing tasks and parse the heterogeneous computing tasks to obtain task requirements.

[0041] The tenant quota and SLA management module is used to determine the tenant to which the heterogeneous computing task belongs and the multi-dimensional tenant attributes corresponding to the tenant based on the heterogeneous computing task.

[0042] The multi-dimensional resource profiling plugin module is used to obtain multi-dimensional indicator data of the working nodes in the distributed computing cluster, filter out multiple candidate working nodes according to the task requirements and the multi-dimensional indicator data of the working nodes, and calculate the multi-dimensional resource profile of each candidate working node according to the multi-dimensional indicator data of the candidate working nodes.

[0043] The tenant quota and priority joint decision-making module is used to calculate the comprehensive scheduling weight of the tenant to which the heterogeneous computing task belongs based on the multi-dimensional tenant attributes.

[0044] The topology-aware module is used to generate at least one set of candidate worker node combinations based on the multi-dimensional resource profile, calculate the corresponding topology score for each candidate worker node combination, and obtain a comprehensive scheduling score by combining the comprehensive scheduling weight and the card type matching degree between the task requirements and the candidate worker node combinations; based on the comprehensive scheduling score, determine the target worker node combination from the at least one set of candidate worker node combinations, and schedule the heterogeneous computing task to the target worker node combination for execution.

[0045] In some embodiments, the system further includes:

[0046] The worker node profile acquisition module is used to collect initial indicator data of worker nodes in the distributed computing cluster.

[0047] The cluster status monitoring module is used to monitor the running status of the worker nodes in the distributed computing cluster.

[0048] The multi-dimensional resource profiling plugin module is used to obtain multi-dimensional indicator data of the worker nodes in the distributed computing cluster based on the initial indicator data collected by the worker node profiling acquisition module and the data monitored by the cluster status monitoring module.

[0049] In some embodiments, the system further includes: an allocation engine module, configured to parse the task profile of the heterogeneous computing task, the task profile including GPU utilization estimation, memory requirements, and memory isolation identifier; determine a target GPU partitioning mode from the following based on the task profile: dedicated GPU mode, multi-instance GPU space partitioning mode, and virtual GPU time partitioning mode; allocate GPU resources to the heterogeneous computing task according to the target GPU partitioning mode, and start the heterogeneous computing task.

[0050] In some embodiments, the topology-aware module is further configured to attempt to atomically bind all worker nodes in the target worker node combination before scheduling the heterogeneous computing task to the target worker node combination. If the binding of any worker node in the target worker node combination fails, a rollback operation is performed on the already bound worker nodes, and the target worker node combination is regenerated until all worker nodes in the target worker node combination are successfully bound.

[0051] In some embodiments, the scheduling framework of the distributed computing cluster is configured with a custom scheduling plugin, which is configured with: a Filter extension point, a Score extension point, and a Permit extension point;

[0052] The Filter extension point is used to call the multi-dimensional resource profiling plugin module to filter out multiple candidate working nodes that meet the resource requirements of heterogeneous computing tasks.

[0053] The Score extension point is used to call the multi-dimensional resource profiling plugin module to calculate the multi-dimensional resource profile of the candidate worker nodes, and combine it with the comprehensive scheduling weight of the tenants in the task calculated by the tenant quota and priority joint decision module and the topology score calculated by the topology awareness module to obtain the comprehensive score of the candidate worker nodes, thereby determining the target worker node combination.

[0054] The Permit extension point is used to call the topology-aware module to perform atomic binding on all worker nodes in the target worker node combination. If the binding of any worker node fails, a rollback operation is performed on the reserved worker nodes. After the atomic binding is successful, the target worker node combination is approved to enter the binding cycle so that the heterogeneous computing tasks can be bound to the target worker node combination for execution.

[0055] The present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, characterized in that the processor, when executing the computer program, implements the multi-tenant dynamic scheduling method for heterogeneous computing tasks described above.

[0056] The present invention also provides a computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements the multi-tenant dynamic scheduling method for heterogeneous computing tasks described above. Attached Figure Description

[0057] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the following detailed description to explain the invention, but do not constitute a limitation thereof. In the drawings:

[0058] Figure 1 This is a flowchart of the steps of a multi-tenant dynamic scheduling method for heterogeneous computing tasks in some embodiments of the present invention.

[0059] Figure 2 This is a schematic diagram of the composition structure of a multi-tenant dynamic scheduling system for heterogeneous computing tasks in some embodiments of the present invention.

[0060] Figure 3 This is a flowchart of the steps of a multi-tenant dynamic scheduling method for heterogeneous computing tasks in some other embodiments of the present invention.

[0061] Figure 4 This is a schematic diagram of the composition structure of a computer device in some embodiments of the present invention. Detailed Implementation

[0062] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0063] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0064] Unless otherwise defined, the technical or scientific terms used in the embodiments of this invention shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0065] As used herein, “parallel” and “perpendicular” include the described situation and situations that are similar to the described situation, within an acceptable range of deviation, which is determined by those skilled in the art taking into account the measurement under discussion and the error associated with the measurement of a particular quantity (i.e., the limitations of the measurement system). For example, “parallel” includes absolute parallelism and approximate parallelism, where an acceptable range of deviation for approximate parallelism may be, for example, within 5°; “perpendicular” includes absolute perpendicularity and approximate perpendicularity, where an acceptable range of deviation for approximate perpendicularity may also be, for example, within 5°.

[0066] This document describes exemplary embodiments with reference to sectional views and / or plan views, which are idealized exemplary drawings. In the drawings, the thickness of layers and regions is enlarged for clarity. Therefore, variations in shape relative to the drawings are contemplated due to, for example, manufacturing techniques and / or tolerances. Therefore, exemplary embodiments should not be construed as limited to the shapes of the regions shown herein, but rather include shape deviations due to, for example, manufacturing processes. Thus, the regions shown in the drawings are schematic in nature, and their shapes are not intended to show the actual shapes of the regions of the device, nor are they intended to limit the scope of the exemplary embodiments.

[0067] With the rapid development of Artificial Intelligence (AI) and high-performance computing technologies, the scale of computing tasks, represented by AI training, AI inference, and model fine-tuning, continues to grow. These heterogeneous computing tasks typically run in multi-tenant shared cluster environments composed of various accelerators such as GPUs, NPUs, and FPGAs. Container orchestration platforms such as Kubernetes have become the mainstream infrastructure for supporting these tasks. In multi-tenant heterogeneous computing clusters, problems such as resource competition among tenants, significant differences in the capabilities of heterogeneous accelerators, and the high sensitivity of distributed tasks to hardware communication topology are intertwined, placing high demands on task scheduling methods.

[0068] To address the above issues, the industry has explored various approaches, and existing technical solutions can be broadly categorized as follows:

[0069] In the area of ​​multi-tenant quota scheduling, some solutions construct a two-level structure of user queues and cluster queues to achieve elastic quota allocation and scheduling ordering based on the estimated task runtime, and support preemption of low-priority tasks by high-priority tasks. Other solutions score and sort nodes based on the availability of resources such as CPU, GPU, and memory, and place tasks accordingly. However, these solutions mainly focus on quota allocation and coarse-grained resource availability assessment, lacking topology affinity awareness of accelerators and high-speed interconnect networks, lacking profiling of the differences in capabilities of heterogeneous accelerator cards, and failing to link decisions with tenant service levels (SLAs), thus failing to guarantee the communication efficiency between the working processes of distributed tasks.

[0070] In the area of ​​accelerator sharing and fine-grained scheduling, some solutions partition and share resources at the GPU memory level, supporting various allocation strategies such as compact placement, uniform distribution, and exclusive use. Other solutions achieve fine-grained fragmentation and reduction of accelerator resources through virtualization logic partitioning and extended scheduler scoring. However, these solutions primarily aim to improve single-card utilization and generally lack hardware topology awareness and tenant priority differentiation. They also lack the atomic batch scheduling (Gang Scheduling) guarantee required for distributed tasks, making it difficult to meet the stringent communication topology requirements of tasks such as distributed training.

[0071] In the area of ​​topology-aware scheduling, some cloud vendors' container services already offer capabilities such as accelerator shared scheduling, accelerator / CPU topology-aware scheduling, and Gang Scheduling. The Kubernetes community has also launched a native topology-aware scheduling solution, supporting the placement of Pod groups in the same topology domain and coordinating with group scheduling policies. Mainstream batch schedulers also provide functions such as Gang scheduling, topology plugins, and network-aware scheduling. However, in such solutions, the various capabilities are usually independent and operate in a decentralized manner: topology awareness is mostly limited to a single Pod or a node's local topology domain and has not yet been extended to cluster-level overall topology optimization for task groups; a multi-dimensional joint decision-making mechanism that integrates tenant service levels, heterogeneous accelerator card capability profiles, and topology optimization has not been formed; and the time-division and space-division partitioning of accelerators are mostly implemented as independent functions without unified decision-making at the scheduling layer.

[0072] In the area of ​​heterogeneous card matching, some solutions select nodes based on the type and quantity of accelerator cards required by the task, combined with global cluster computing power data, and establish a mutual trust network between nodes. However, such solutions mainly perform static card matching and node selection, lacking real-time resource profiling and topology status awareness, making it impossible to perform multi-dimensional joint optimization, and difficult to dynamically adjust scheduling decisions based on changes in real-time cluster load and topology status.

[0073] In summary, existing scheduling technologies generally optimize only from a single or few dimensions, such as quota management, resource availability, accelerator sharing, topology co-location, or card type matching. They lack effective integration of factors such as tenant service level differentiation, heterogeneous accelerator card capability profiling, multi-level hardware topology awareness, atomic scheduling of batch tasks, and hybrid time-division / space-division accelerator partitioning. This makes it difficult to achieve multi-objective overall optimization of scheduling quality, resource utilization, and tenant fairness in multi-tenant heterogeneous computing power clusters. Therefore, there is an urgent need in this field for a multi-dimensional, integrated dynamic scheduling scheme for heterogeneous computing tasks.

[0074] In order to at least alleviate or solve one of the aforementioned technical problems, the present invention provides a multi-tenant dynamic scheduling method, system, and device for heterogeneous computing tasks.

[0075] Figure 1 This is a flowchart of the steps of a multi-tenant dynamic scheduling method for heterogeneous computing tasks in some embodiments of the present invention.

[0076] This invention provides a multi-tenant dynamic scheduling method for heterogeneous computing tasks, which is applied to a distributed computing cluster, including multiple worker nodes.

[0077] A distributed computing cluster is a distributed computing system that interconnects a large number of independent worker nodes through a high-speed network. A distributed computing cluster can break down a large, complex heterogeneous computing task into multiple subtasks, which are then distributed to multiple worker nodes for synchronous parallel processing. This significantly improves computing speed and processing power, and offers advantages such as high availability, elastic scalability, and heterogeneous collaboration.

[0078] The scheduling method of this invention can be applied to various specific distributed computing cluster systems, such as Kubernetes (K8s) clusters, Docker Swarm clusters, Slurm clusters, or Apache Mesos clusters. This invention will be described in detail using a Kubernetes cluster as an example.

[0079] In a distributed computing cluster, worker nodes are the actual execution units of the cluster. For example, in a Kubernetes (K8s) cluster, worker nodes are machines that run containerized applications (Pods). A Pod is the smallest deployable and schedulable computing unit in a K8s cluster. A Pod can contain one or more closely related containers. Each worker node runs core components such as Kubelet and Kube-proxy, responsible for receiving scheduling tasks from the K8s cluster's control plane (Master), starting and managing containers, and periodically reporting the node's status.

[0080] In some embodiments, such as Figure 1 As shown, the present invention provides a multi-tenant dynamic scheduling method for heterogeneous computing tasks, comprising the following steps:

[0081] Step S1: Obtain heterogeneous computing tasks, parse the heterogeneous computing tasks to obtain task requirements, and determine the tenant to which the heterogeneous computing tasks belong and the multi-dimensional tenant attributes corresponding to the tenant.

[0082] It should be noted that heterogeneous computing tasks refer to various computing workloads that are completed collaboratively by two or more different types of processors (such as CPU, GPU, FPGA, ASIC / NPU, DSP, etc.). In other words, any task that involves the collaboration or allocation of computing power from multiple architectures is considered a heterogeneous computing task.

[0083] Optionally, heterogeneous computing tasks can include AI inference, AI training and fine-tuning, data processing and batch processing, and high-performance computing (HPC) tasks. High-performance computing tasks include scientific computing such as weather simulation, fluid dynamics calculations, gene sequencing, and molecular dynamics. Data processing and batch processing tasks include large-scale database queries, graph computation, and real-time streaming data processing.

[0084] This invention uses AI training tasks as a specific example of heterogeneous computing tasks for illustration. AI training tasks are a typical type of heterogeneous computing task, which highly depends on the collaborative work of GPU clusters or NPU clusters (heterogeneous computing power) and CPUs (responsible for data preprocessing and scheduling).

[0085] Optionally, task requirements may include data requirements, task type, task objectives, resource and storage requirements, accuracy and energy efficiency requirements, task execution dependencies, high-performance computing power requirements, time and budget constraints, and communication bandwidth requirements. For example, for AI training tasks, data requirements include massive and diverse data sources, and whether these data sources need to undergo rigorous preprocessing such as deduplication, denoising, and format standardization, and whether they include boundary cases and hard examples to ensure the model's generalization ability. Task type requirements include determining the type of AI training task, such as Natural Language Processing and Generation (NLP), Computer Vision (CV), Prediction and Regression Analysis, Classification and Clustering, Reinforcement Learning and Decision Support, or Multimodal and Interactive Tasks. High-performance computing power requirements may include the necessary AI accelerators such as GPUs and TPUs. Time requirements involve strict and refined management of computing resource scheduling, mixed-precision training, and memory optimization.

[0086] It is understandable that multi-dimensional tenant attributes are the core basis for the scheduling system to perform refined resource governance and differentiated services. Optionally, multi-dimensional tenant attributes include at least one of the following: Service Level Agreement (SLA) level, resource quota limit, historical usage efficiency statistics, and task urgency label.

[0087] SLA levels represent a tenant's "identity value." For example, paid VIP clients (core financial trading systems) have high SLA levels, while free test users have low SLA levels.

[0088] Historical usage efficiency statistics (FairShare) represent a tenant's "historical performance." If a tenant has consistently wasted resources in the past, their weight is reduced; if a tenant has consistently been efficient, they are rewarded and their weight is increased.

[0089] The task urgency label represents the "time sensitivity" of the current task.

[0090] Step S2: Obtain multi-dimensional indicator data of worker nodes in the distributed computing cluster, and select multiple candidate worker nodes based on task requirements and the multi-dimensional indicator data of worker nodes.

[0091] Optionally, in some embodiments, the working node is configured with a heterogeneous accelerator card. The heterogeneous accelerator card includes at least one of a graphics processing unit (GPU), a neural network processor (NPU), and a field-programmable gate array (FPGA). In this case, the multidimensional metric data includes at least one of the following: accelerator card model, memory capacity, node internal interconnect link status, inter-node communication network topology, accelerator card compute unit utilization, and allocated memory.

[0092] As previously known, task requirements may include the amount of resources required for the task (e.g., how many GPUs and how much memory are needed), the matching degree of the computing type (e.g., it must be a GPU node that supports FP16), data locality (e.g., which rack the data is stored in), and specific QoS constraints (e.g., the maximum acceptable latency), etc.

[0093] This step essentially establishes a filtering logic based on the task requirements of the AI ​​training task and the multi-dimensional indicators of the worker nodes, selecting suitable candidate worker nodes from multiple worker nodes. These candidate worker nodes can be preliminarily determined to be suitable for executing the AI ​​training task.

[0094] Optionally, the filtering logic may include specific filtering methods such as resource capacity filtering, availability and reliability filtering, and QoS and constraint filtering.

[0095] Resource capacity filtering can, for example, eliminate worker nodes whose available resources (such as remaining memory and idle computing power) are less than the requirements of the corresponding task, ensuring that candidate worker nodes have sufficient physical resources to carry the task.

[0096] Availability and reliability filtering can eliminate work nodes that are unstable, about to go offline for maintenance, or have potential hardware failures, ensuring that tasks can be executed reliably.

[0097] QoS and constraint filtering can filter out working nodes that do not meet the conditions based on the maximum latency tolerance of the task or specific hardware requirements.

[0098] Step S3: Calculate the multidimensional resource profile of each candidate worker node based on the multidimensional indicator data of the candidate worker nodes, and generate at least one combination of candidate worker nodes based on the multidimensional resource profile.

[0099] In this step, calculating the multi-dimensional resource profile can be understood as assigning multiple capability tags to candidate working nodes. Optionally, in some embodiments, the multi-dimensional resource profile includes: card performance vector, memory fragmentation rate, topology matrix, and current computing power utilization.

[0100] The card type performance vector includes a combination of values ​​representing multiple dimensions such as computing power, memory capacity, interconnect bandwidth, and architectural features. When the values ​​of the dimensions representing computing throughput and interconnect bandwidth in the vector are higher than preset thresholds, the card type is determined to be a high-performance accelerator card.

[0101] For example, a card performance vector typically includes the following dimensions: computing power, storage, communication, and architectural features. For example, it can be represented as: [Computing power: 90, video memory: 80GB, interconnect bandwidth: 600GB / s, architecture generation: 5].

[0102] Video memory fragmentation rate refers to the proportion of a large number of scattered and discontinuous free blocks of space remaining after a long period of dynamic allocation and release of GPU video memory (VRAM).

[0103] A topology matrix is ​​a diagram that describes the physical connections and relationships between various hardware devices (such as GPUs, CPUs, and network cards) within a cluster.

[0104] Computational utilization can be understood on two levels: GPU utilization (a surface-level metric) and Model FLOPs Utilization (MFU) (a core, realistic metric). GPU utilization refers to the percentage of time the GPU spends executing kernel operations within a past sampling period. Model FLOPs Utilization (MFU) refers to the ratio of observed actual throughput to the theoretical maximum throughput of the system at peak floating-point operations per second (FLOPS), and is the best indicator for measuring the efficiency of computational utilization during large model training.

[0105] Optionally, in some embodiments, a multi-dimensional resource profile is calculated based on multi-dimensional indicator data, specifically including: determining the card type performance vector based on the accelerator card model; calculating the memory fragmentation rate based on the memory capacity and the allocated memory; constructing a topology matrix based on the internal interconnection link status of the node; and evaluating the current computing power utilization rate based on the accelerator card computing unit occupancy rate.

[0106] In this step, the topology matrix determines which worker node combinations the task should be scheduled to (ensuring fast communication); after scheduling, the memory fragmentation rate needs to be monitored (ensuring no waste or interruption of resources); finally, by improving the management level of these two aspects, the current computing power utilization (MFU) will be improved from a generally inefficient state (e.g., 30%) to a highly efficient state (e.g., over 70%), thereby solving the core pain point of multi-tenant heterogeneous clusters.

[0107] Step S4: Calculate the comprehensive scheduling weight of the tenant to which the heterogeneous computing task belongs based on the multi-dimensional tenant attributes.

[0108] For example, in some embodiments, the comprehensive scheduling weight of the tenant to which the heterogeneous computing task belongs is calculated based on the multidimensional tenant attributes, including: calculating the comprehensive scheduling weight by weighting according to the SLA level, historical usage efficiency statistics and task urgency label.

[0109] Furthermore, the scheduling weight of the tenant to which the heterogeneous computing task belongs is calculated based on the multi-dimensional tenant attributes. Specifically, this includes: obtaining the tenant's Service Level Agreement (SLA) level, historical resource utilization efficiency, and current task urgency label; normalizing the SLA level, historical resource utilization efficiency, and urgency label to obtain the corresponding SLA weight coefficient, efficiency penalty / reward coefficient, and urgency coefficient; and weighting the above three coefficients according to a preset weighted summation formula to obtain the tenant's comprehensive scheduling weight.

[0110] Among them, the overall scheduling weight is positively correlated with the SLA level and urgency label, and is dynamically reduced or increased based on historical resource utilization efficiency.

[0111] This step is equivalent to calculating a comprehensive score by taking into account the tenant's SLA level, historical usage efficiency, and current urgency. Tenants with higher scores are more likely to secure GPU computing resources.

[0112] Step S5: Calculate the corresponding topology score for each candidate worker node combination, and combine the comprehensive scheduling weight and the card type matching degree between the task requirements and the candidate worker node combination to obtain the comprehensive scheduling score.

[0113] Alternatively, in another alternative implementation, the topology score can be calculated in the following manner (i.e., an alternative to Equation (2)): obtain the physical interconnection type between each node in the candidate working node combination (such as NVLink, PCIe, cross NUMA nodes, etc.); convert the physical interconnection type into the corresponding communication score according to the preset communication bandwidth mapping rule (for example, NVLink direct connection is assigned 100 points, same PCIe switch is assigned 60 points, cross CPU bus is assigned 10 points); accumulate or weighted sum the communication scores between all nodes in the candidate working node combination to obtain the comprehensive topology score of the candidate working node combination.

[0114] Optionally, in some embodiments, the task requirements include the target accelerator card model; the multidimensional indicator data includes the actual accelerator card model; and the card type matching degree is calculated based on the target accelerator card model and the actual accelerator card model.

[0115] Optionally, the formula for calculating the comprehensive scheduling score is: Comprehensive scheduling score = (topology score × topology weight coefficient) + (comprehensive scheduling weight × service priority coefficient) + (card type matching degree × hardware adaptation coefficient).

[0116] Among them, the topology weight coefficient, business priority coefficient, and hardware adaptation coefficient are all real numbers greater than 0, and the sum of the three is preferably 1 (normalized), for example, 0.4, 0.3, and 0.3 respectively; preferably, the topology score and card type matching degree are normalized to the [0,1] interval according to the theoretical maximum value in the candidate working node combination before participating in the above calculation.

[0117] This step can take into account the underlying hardware physical characteristics (topology), upper-layer business requirements (SLA weight), and heterogeneous computing power matching.

[0118] Step S6: Based on the comprehensive scheduling score, determine the target working node combination from at least one set of candidate working node combinations, and schedule the heterogeneous computing tasks to the target working node combination for execution.

[0119] In this step, all candidate worker node combinations can be traversed, and one or more combinations with the highest final comprehensive scheduling score can be selected as the optimal target combination to simultaneously ensure the communication efficiency, business priority, and computing power matching of the task.

[0120] In the development directions of multi-tenant quota scheduling, GPU shared scheduling, topology-aware scheduling, and heterogeneous card type matching, existing technologies mainly focus on single-dimensional or two-dimensional optimization, and no scheduling scheme has yet emerged that jointly optimizes three dimensions: "tenant SLA level, GPU topology affinity, and heterogeneous card type capability differences." The essence of this invention lies precisely in this. The technical solution proposed in this invention not only focuses on quota allocation but also on topology awareness and card type matching, possessing the ability to simultaneously incorporate tenant SLA level, GPU topology affinity, and heterogeneous card type capability differences into the scheduling decision. This avoids scheduling results that are optimal in one dimension but suboptimal in others.

[0121] In some embodiments, step S6, generating at least one set of candidate working node combinations based on the multidimensional resource profile, specifically includes:

[0122] Step S61: Based on the multi-dimensional resource profile, select candidate work nodes that meet the preset conditions from multiple work nodes.

[0123] As mentioned above, the multi-dimensional resource profile is to tag the work nodes with capability labels. In this step, a screening is carried out based on the capability labels and preset conditions.

[0124] It is understood that the preset conditions refer to the filtering rules used to evaluate whether candidate working nodes have the basic ability to perform the current AI training task. Those skilled in the art will understand that the specific values ​​or rules of the preset conditions can be flexibly adjusted according to the device type, task characteristics, or user habits, and this invention does not impose a unique limitation on them.

[0125] For example, in one specific implementation, the preset conditions can be set to at least one of the following: the overall health score of the candidate working node is greater than a preset score threshold; the available video memory capacity of the candidate working node is greater than or equal to the video memory requirement declared by the AI ​​training task; the accelerator card model of the candidate working node is consistent with the target accelerator card model requested by the AI ​​training task; and the video memory allocation fragmentation rate of the candidate working node is less than a preset fragmentation rate threshold.

[0126] Step S62: Based on the Gang Scheduling strategy, select a combination of nodes from the candidate worker nodes that meets the requirements for the number of worker nodes and card type for heterogeneous computing tasks.

[0127] Existing scheduling methods often lack guarantees for group scheduling when handling distributed tasks, which can easily lead to task deadlock or resource fragmentation. Step S62 of this invention introduces a group scheduling strategy and combines it with dual matching of card type and quantity in heterogeneous clusters to ensure the atomicity of task startup and strong consistency of hardware specifications in the initial stage of scheduling, laying a safe foundation for subsequent topology optimization.

[0128] AI training tasks often require dozens or even hundreds of GPUs, and these GPUs must be running simultaneously for gradient synchronization. Therefore, when searching for "node combinations," the Gang Scheduling algorithm should be used to find all sets of nodes that simultaneously meet the requirements of "quantity" and "card type," ensuring that these nodes can be atomically (once) allocated to the AI ​​training task.

[0129] Step S63: Obtain the physical communication link status between each working node in the distributed computing cluster, parse the cluster communication topology diagram, and calculate the topology connectivity score within each node combination based on the cluster communication topology diagram.

[0130] In this step, obtaining the physical communication link status allows us to understand the "physical distance" and "bandwidth limit" of the underlying hardware. Parsing the cluster communication topology map allows us to quantify the connections between nodes using mathematical models or graph algorithms. Calculating the topology connectivity score within each node combination converts the physical layer hardware connections into comparable "numerical scores," facilitating the analysis of these simple numerical scores.

[0131] Topology connectivity is a metric that measures how tightly a group of worker nodes are physically connected and how fast data is transmitted. The higher the topology connectivity, the higher the overall efficiency (MFU) of the AI ​​training task.

[0132] Step S64: Node combinations whose topology connectivity scores meet a preset threshold are identified as candidate working node combinations. If no node combination meets the preset threshold, the node combination with the highest topology connectivity score is identified as a candidate working node combination.

[0133] AI large-scale model training tasks are extremely sensitive to network communication. If the topology connectivity score of a combination of worker nodes is too low, the GPU will spend most of its training time in a state of "waiting for network data transmission," resulting in serious waste of computing power and extremely low MFU (Mean Functional Fusion).

[0134] If no combination of working nodes meets the preset threshold, a degraded scheduling strategy is triggered: the combination of nodes with the highest topology connectivity score is selected as a candidate working node combination; or, the preset threshold is dynamically lowered by a preset step size until at least one candidate working node combination is selected. This setting is to prevent AI training tasks from waiting indefinitely in the queue due to the inability to gather sufficient resources, which could eventually lead to timeout failure or forced cancellation by the user.

[0135] In some embodiments, before scheduling the heterogeneous computing task to the target worker node combination in step S6, the scheduling method further includes: attempting to atomically bind all worker nodes in the target worker node combination; if the binding of any worker node in the target worker node combination fails, performing a rollback operation on the bound worker nodes and regenerating the target worker node combination until all worker nodes in the target worker node combination are successfully bound.

[0136] In multi-tenant and heterogeneous computing scenarios, this step employs a full-resource atomic reservation and rollback mechanism to ensure the smooth execution of complex tasks (such as distributed AI training and cross-node parallel computing).

[0137] In heterogeneous computing and multi-tenant shared resource environments, this atomic binding mechanism can prevent resource fragmentation and deadlock, ensure the coordination and SLA of distributed tasks, and maintain the fairness and isolation of multi-tenant environments.

[0138] Heterogeneous computing tasks typically require multiple nodes to work collaboratively. If the scheduler only successfully binds some worker nodes without atomicity checks, some resources will be "dangled." Other tasks may be unable to be scheduled due to insufficient remaining resources, but the already bound nodes are waiting for the missing nodes, ultimately leading to system resource fragmentation or even deadlock. Full binding is required for the binding to take effect, ensuring that resources are either fully available or completely released.

[0139] For heterogeneous computing tasks that require close collaboration, the absence of any single computing node will cause the entire job to fail. Through atomic binding, the scheduler confirms that all necessary resources (such as specific GPU models and high-speed network bandwidth) are in place before the task is actually issued for execution, avoiding mid-process crashes due to resource shortages after task startup and ensuring the quality of service (SLA).

[0140] In a multi-tenant environment, resource quotas are strictly controlled. If binding fails but is not rolled back, tenant A may unintentionally occupy "invalid resources" exceeding its quota, thereby squeezing tenant B's available resource pool. Atomicity rollback ensures that only tasks that can actually run will have their quotas deducted, guaranteeing fairness among tenants.

[0141] This defensive scheduling strategy, through the mechanism of "full binding and failure rollback," perfectly meets the requirements of strong resource consistency in distributed heterogeneous computing and is a key step for multi-tenant computing platforms to ensure system stability and resource utilization.

[0142] In some embodiments, based on the foregoing embodiments, after all worker nodes in the target worker node combination are successfully bound, the scheduling method further includes:

[0143] Step S7: Analyze the task profile of the heterogeneous computing task. The task profile includes GPU utilization prediction, memory requirements, and memory isolation markers.

[0144] This step is equivalent to "assessing" the heterogeneous computing task before actually distributing it to each worker node. GPU utilization estimation is used to determine whether the task requires full computing power (such as large model pre-training) or has low computing power requirements but is sensitive to video memory (such as small model inference, development testing). Video memory requirements are used to determine how many GB of video memory the task needs, which is the basis for deciding whether to partition the task and the partitioning specifications. Video memory isolation flag: used to indicate whether the task is sensitive to "noisy neighbors" and whether hardware-level video memory isolation is required.

[0145] This step transforms abstract task requirements into metrics that can be quantified by the scheduler, providing data support for subsequent decision-making.

[0146] Step S8: Based on the task profile, determine the target GPU partitioning mode from the following options: dedicated GPU mode, multi-instance GPU (MIG) space-sharing mode, and virtual GPU (vGPU) time-sharing mode. Optionally, before determining the target GPU partitioning mode, determine the set of available partitioning modes based on the hardware architecture type of the GPU on the target worker node.

[0147] This step is equivalent to matching the most suitable resource supply method based on the task profile in step S7.

[0148] The dedicated GPU mode corresponds to either "high utilization prediction" or "strong isolation requirements." It is suitable for pre-training models with hundreds of billions of parameters or for online inference that is extremely sensitive to latency, with no performance loss.

[0149] The Multi-Instance GPU (MIG) spatial division mode corresponds to "medium utilization prediction" and "requires hardware-level isolation". MIG divides the physical GPU into up to 7 independent instances (with independent video memory, L2 cache and computing units) at the hardware level, which is suitable for multi-tenant security-sensitive scenarios and small to medium model inference.

[0150] Virtual GPU (vGPU) time-sharing mode is suitable for "low utilization forecasts" and "low isolation requirements". It achieves time-sharing multiplexing of multiple tasks through software layers (such as CUDA time-slice rotation), which is highly flexible and suitable for development, testing and lightweight inference.

[0151] Step S9: Allocate GPU resources to the heterogeneous computing task according to the target GPU partitioning mode and start the heterogeneous computing task. During task execution, continuously monitor the actual GPU utilization and memory pressure. If the deviation between the actual performance profile and the task performance profile exceeds a preset threshold, trigger a dynamic switch of the target GPU partitioning mode (e.g., downgrade from MIG to vGPU, or upgrade from vGPU to dedicated GPU).

[0152] This step, after matching the resource supply method in step S8, involves performing resource locking and task initiation to implement the decision. For example, it calls the underlying driver (such as NVIDIA Device Plugin) or virtualization component to complete the actual allocation of video memory and computing power, and schedules the heterogeneous computing tasks to run on the partitioned GPU instance.

[0153] In some embodiments, in step S8, based on the task profile, the target GPU partitioning mode is determined from the exclusive GPU mode, the multi-instance GPU (MIG) spatial partitioning mode, and the virtual GPU (vGPU) time-division sharing mode, specifically including:

[0154] When the estimated GPU utilization is higher than the preset utilization threshold, or the memory isolation flag indicates that strong isolation is required, the heterogeneous computing task is allocated either a dedicated GPU mode or a MIG space-division mode. For example, when the estimated GPU utilization is higher than the preset utilization threshold, or the memory isolation flag indicates that strong isolation is required, the heterogeneous computing task is allocated a multi-instance GPU (MIG) space-division mode; if the GPU hardware of the current working node does not support the multi-instance GPU (MIG) space-division mode, the dedicated GPU mode is allocated as a fallback.

[0155] When the estimated GPU utilization is lower than the preset utilization threshold and the memory isolation flag indicates that sharing is allowed, vGPU time-sharing mode is allocated to heterogeneous computing tasks.

[0156] This step precisely routes development and testing tasks, lightweight inference, and other tasks to vGPU time-sharing mode by using the condition "GPU utilization is estimated to be below the threshold," thus avoiding wasted computing power. Simultaneously, by using the condition "memory isolation flag indicates strong isolation is required," critical business or online inference services are routed to MIG or dedicated mode. MIG provides hardware-level memory and L2 cache isolation, completely eliminating the "noisy neighbor" effect and memory overrun risk common in vGPU time-sharing mode.

[0157] This step transforms the tedious task of manually judging and configuring GPU mode, which originally required maintenance personnel, into automatic decision-making driven by "task profiles," realizing dynamic matching of computing power supply and business needs, and achieving automation and intelligence of scheduling strategies.

[0158] In some embodiments, step S9, allocating GPU resources to heterogeneous computing tasks according to the target GPU partitioning mode, includes:

[0159] Step S91: Report the number and availability of allocable GPU resource instances exposed by each worker node in the target worker node combination under the target GPU partitioning mode to the distributed computing cluster through the node device plugin.

[0160] Step S92: On the same worker node in the target worker node combination, run MIG space division mode and vGPU time division sharing mode respectively to support the hybrid deployment of heterogeneous GPU resources.

[0161] Optionally, different physical GPUs can be used to run MIG spatial division mode and vGPU time-division sharing mode respectively. For example, this process can specifically include: allocating instance resources in MIG spatial division mode to the first heterogeneous computing task, and allocating instance resources in vGPU time-division sharing mode to the second heterogeneous computing task. The first heterogeneous computing task and the second heterogeneous computing task are two different heterogeneous computing tasks. This step allows different physical GPUs to be divided into different resource pools (one part for high-security isolation, and one part for high-concurrency sharing) within a single worker node according to hardware topology and business requirements, greatly improving the scheduling flexibility and resource utilization of the heterogeneous cluster.

[0162] This invention also provides a multi-tenant dynamic scheduling system for heterogeneous computing tasks, applied to a distributed computing cluster, wherein the scheduling system is built into the distributed computing cluster, which includes multiple worker nodes.

[0163] Figure 2 This is a schematic diagram of the composition structure of a multi-tenant dynamic scheduling system for heterogeneous computing tasks in some embodiments of the present invention.

[0164] like Figure 2 As shown, the present invention provides a multi-tenant dynamic scheduling system for heterogeneous computing tasks, comprising: a task analysis module, a tenant quota and SLA management module 30, a multi-dimensional resource profiling plugin module 40, a tenant quota and priority joint decision-making module 50, and a topology sensing module 60.

[0165] The task analysis module is used to acquire heterogeneous computing tasks and parse them to obtain task requirements.

[0166] The tenant quota and SLA management module 30 is used to determine the tenant to which the heterogeneous computing task belongs and the multi-dimensional tenant attributes corresponding to the tenant based on the heterogeneous computing task.

[0167] The multidimensional resource profiling plugin module 40 is used to obtain multidimensional indicator data of worker nodes in the distributed computing cluster, filter out multiple candidate worker nodes based on task requirements and multidimensional indicator data of worker nodes, and calculate the multidimensional resource profile of each candidate worker node based on the multidimensional indicator data of the candidate worker nodes.

[0168] The tenant quota and priority joint decision module 50 is used to calculate the comprehensive scheduling weight of the tenant to which the heterogeneous computing task belongs based on the multi-dimensional tenant attributes.

[0169] The topology-aware module 60 is used to generate at least one set of candidate worker node combinations based on the multi-dimensional resource profile, calculate the corresponding topology score for each candidate worker node combination, and obtain the comprehensive scheduling score by combining the comprehensive scheduling weight and the card type matching degree between the task requirements and the candidate worker node combination; based on the comprehensive scheduling score, determine the target worker node combination from at least one set of candidate worker node combinations, and schedule the heterogeneous computing task to the target worker node combination for execution.

[0170] In some embodiments, the multi-tenant dynamic scheduling system for heterogeneous computing tasks of the present invention further includes: a worker node profile acquisition module 10 and a cluster status monitoring module 20.

[0171] Among them, the worker node profile acquisition module 10 is used to collect the initial indicator data of worker nodes in the distributed computing cluster.

[0172] The cluster status monitoring module 20 is used to monitor the running status of worker nodes in the distributed computing cluster.

[0173] The multi-dimensional resource profiling plugin module 40 is also used to obtain multi-dimensional indicator data of the working nodes in the distributed computing cluster based on the initial indicator data collected by the working node profiling acquisition module 10 and the data monitored by the cluster status monitoring module 20.

[0174] In some embodiments, the multi-tenant dynamic scheduling system for heterogeneous computing tasks according to the present invention further includes an allocation engine module 70. The allocation engine module 70 is used to parse the task profile of the heterogeneous computing task, the task profile including GPU utilization estimation, memory requirements, and memory isolation identifier; based on the task profile, determine the target GPU partitioning mode from exclusive GPU mode, multi-instance GPU (MIG) space-division mode, and virtual GPU (vGPU) time-division sharing mode; allocate GPU resources to the heterogeneous computing task according to the target GPU partitioning mode, and start the heterogeneous computing task.

[0175] In some embodiments, the topology-aware module 60 is further configured to attempt to atomically bind all worker nodes in the target worker node combination before scheduling heterogeneous computing tasks to the target worker node combination. If the binding of any worker node in the target worker node combination fails, a rollback operation is performed on the bound worker nodes, and the target worker node combination is regenerated until all worker nodes in the target worker node combination are successfully bound.

[0176] In some embodiments, the scheduling framework of the distributed computing cluster is configured with a custom scheduling plugin, which includes: a Filter extension point, a Score extension point, and a Permit extension point.

[0177] The Filter extension point is used to call the multi-dimensional resource profiling plugin module 40 to filter out multiple candidate working nodes that meet the resource requirements of heterogeneous computing tasks.

[0178] The Score extension point is used to call the multi-dimensional resource profile plugin module 40 to calculate the multi-dimensional resource profile of the candidate working nodes, and combine it with the comprehensive scheduling weight of the tenant of the task calculated by the tenant quota and priority joint decision module 50 and the topology score calculated by the topology perception module 60 to obtain the comprehensive score of the candidate working nodes, thereby determining the target working node combination.

[0179] The Permit extension point is used to call the topology-aware module 60 to perform atomic binding on all worker nodes in the target worker node combination. If the binding of any worker node fails, a rollback operation is performed on the reserved worker nodes. After the atomic binding is successful, the target worker node combination is approved to enter the binding cycle so that the heterogeneous computing tasks can be bound to the target worker node combination for execution.

[0180] In this embodiment of the invention, the Permit extension point in the K8s scheduling framework undertakes the important task of "atomic binding and rollback", successfully linking the aforementioned "multi-dimensional resource profiling", "tenant quota", "topology awareness" and "atomic binding" into a complete scheduling closed loop.

[0181] As described above, this invention provides a multi-dimensional resource profiling plugin and a real-time GPU node profiling generation method. Specifically, a custom plugin is registered at the Filter and Score extension points of the K8s Scheduling Framework to generate a real-time profiling vector containing the GPU type performance for each GPU node. NVLink topology matrix, memory fragmentation rate Computing power utilization The multi-dimensional profile is generated, and the profile data is stored and updated through the GPU Node Profile CRD.

[0182] This invention also provides a joint decision-making model for tenant quotas and priorities, specifically based on tenant SLA level scores. Historical usage efficiency score Task urgency score The three factors are used to calculate the overall scheduling weight using equation (1). In the Score stage, it is combined with node profile scoring to achieve joint optimization of tenant and resource dimensions.

[0183] This invention also provides a topology-aware Gang Scheduling algorithm for determining and executing target worker node combinations. Specifically, for distributed training tasks, candidate worker node combinations are generated during the Permit phase, and the topology score is calculated using equation (2). The comprehensive scheduling score is calculated using equation (3). The optimal node combination is selected by equation (4). This enables atomic scheduling that optimizes the overall topology of PodGroups at the cluster level.

[0184] This invention also proposes a time-division / space-division hybrid partitioning engine and an automatic partitioning mode selection method. Based on the estimated GPU utilization and memory requirements of the task, the GPU partitioning mode is automatically selected: high-utilization tasks are allocated dedicated GPUs or MIG space-division instances, low-utilization tasks are allocated vGPU time-division shared instances, and mixed deployment scenarios support the coexistence of MIG instances and vGPU instances on the same node. The partitioning decision exposes allocable GPU resources to Kubernetes through the Device Plugin.

[0185] This invention also proposes a three-dimensional joint optimization comprehensive scheduling scoring method, which uses equation (3) to calculate tenant weights. Topology score Card type performance Memory fragmentation rate GPU utilization Five factors are multiplicatively coupled to achieve three-dimensional joint optimization scheduling and scoring, ensuring that the highest score is obtained when all three dimensions are satisfied simultaneously.

[0186] The technical solution of the present invention will now be described in conjunction with specific embodiments.

[0187] Example 1

[0188] like Figure 2 As shown, this invention provides a multi-tenant dynamic scheduling system for heterogeneous computing tasks. Logically, the system can be divided into a data acquisition layer, a scheduling decision layer, and a scheduling execution layer. These layers communicate and collaborate via data flow to complete a closed loop from acquiring AI training tasks to executing them.

[0189] The data acquisition layer includes: a task analysis module, a worker node profile acquisition module 10, a tenant quota and SLA management module 30, and a cluster status monitoring module 20. The scheduling decision layer includes a multi-dimensional resource profile plugin module 40, a tenant quota and priority joint decision module 50, and a topology awareness module 60. The scheduling execution layer includes an allocation engine module 70 and an extension point module 80.

[0190] The worker node profile acquisition module 10 includes a GPU node profile collector, which is used to collect real-time multi-dimensional indicators of each GPU node in the cluster, including GPU card type, video memory capacity, NVLink connection status, IB network topology, GPU SM utilization rate, video memory allocation fragmentation rate, etc., to provide underlying data support for scheduling decisions.

[0191] The upstream interface of the GPU node profiling collector connects to the GPU node's DCGM / NVML metric interface and the IB network topology discovery service. The downstream interface writes the collected data to the GPU Node Profile CRD for the multi-dimensional resource profiling plugin in the multi-dimensional resource profiling plugin module 40 to read. At the same time, it provides raw metric data to the cluster status monitoring module 20.

[0192] The tenant quota and SLA management module 30 is used to maintain tenant-side information such as SLA level definition (high priority / medium priority / low priority), resource quota limit, historical usage efficiency statistics, and task urgency labels, providing tenant-dimensional input for the joint decision-making model.

[0193] The upstream interface of the tenant quota and SLA management module 30 connects to the quota and SLA configuration interface of the tenant management platform. The downstream interface of the tenant quota and SLA management module 30 provides tenant information to the joint decision-making model of tenant quota and priority. The tenant quota and SLA management module 30 works in conjunction with the cluster status monitoring module 20 to obtain historical usage data of tenants for efficiency scoring.

[0194] The cluster status monitoring module 20 is used to monitor the resource usage status, Pod running status, GPU utilization trend, etc. of each node in the cluster in real time, providing a global view of the cluster for scheduling decisions.

[0195] The upstream of the cluster status monitoring module 20 obtains data from the GPU node profiling collector and the K8s API Server, while the downstream of the cluster status monitoring module 20 provides global cluster status information to the multi-dimensional resource profiling plugin to assist in scheduling decisions.

[0196] The multi-dimensional resource profiling plugin module 40 includes a multi-dimensional resource profiling plugin used to generate a real-time multi-dimensional resource profile for each GPU node based on data from the GPU node profiling collector and the cluster status monitoring module 20. This profile includes a card performance vector (normalized computing power score), an NVLink topology matrix (direct connection relationships and bandwidth between GPU pairs), memory fragmentation rate, and current computing power utilization. The multi-dimensional resource profiling plugin is registered on the Filter and Score extension points of the K8s Scheduling Framework. During the filtering phase, nodes that do not meet task requirements are excluded, and during the optimization phase, candidate nodes are scored.

[0197] The upstream of the multi-dimensional resource profiling plugin reads GPU Node Profile CRD and cluster status data, while the downstream of the multi-dimensional resource profiling plugin provides node profiles and scoring results to the topology-aware module 60.

[0198] The tenant quota and priority joint decision-making model calculates the comprehensive scheduling weight for each tenant based on three factors: tenant SLA level, historical usage efficiency, and task urgency. This weight is combined with the node profile score during the Score phase to ensure that tasks from tenants with high SLA levels receive priority scheduling during resource contention, while also taking into account resource utilization efficiency and task timeliness.

[0199] The upstream of the tenant quota and priority joint decision-making model obtains tenant information from the tenant quota and SLA management module 30, and the downstream of the tenant quota and priority joint decision-making model outputs the comprehensive scheduling weight to the topology perception module 60 as the tenant dimension factor for joint scoring.

[0200] The topology-aware module 60 can employ the topology-aware Gang Scheduling algorithm to achieve atomic scheduling of distributed training tasks (such as PyTorchJob / TFJob) during the Permit phase. The topology-aware GangScheduling algorithm first generates candidate worker node combinations, calculates the topology score (maximizing the number of NVLink direct connections + minimizing the number of IB hops) for each candidate worker node combination, and then performs a comprehensive ranking based on tenant weights and card type matching. The optimal node combination is selected for atomic binding, and all worker bindings are rolled back if any worker binding fails.

[0201] The upstream of the topology sensing module 60 receives the node scores from the multi-dimensional resource profiling plugin and the weight output of the tenant joint decision model. The downstream of the topology sensing module 60 transmits the scheduling decision to the extension point module 80 for binding.

[0202] The allocation engine module 70 includes a time-division / space-division hybrid allocation engine, used to automatically select the GPU allocation mode based on the task profile (GPU utilization estimation, memory requirements, and whether memory isolation is needed): high GPU utilization tasks are allocated dedicated GPUs or MIG space-division instances, low GPU utilization tasks are allocated vGPU time-division shared instances, and mixed deployment scenarios support the coexistence of MIG instances and vGPU instances on the same node. The allocation decision exposes allocable GPU resources to Kubernetes through the Device Plugin. After the scheduler completes the GPU allocation mode decision, the device plugin on the target worker node exposes the corresponding allocable GPU resources to the Kubernetes cluster.

[0203] The upstream of the allocation engine module 70 receives task allocation instructions from the scheduling decision layer, and the downstream of the allocation engine module 70 exposes the partitioned GPU resource view to the K8s Scheduling Framework extension module through the Device Plugin.

[0204] Extension point module 80 is used as the entry point of the scheduling execution layer, registering the above modules to the corresponding extension point of K8s SchedulingFramework: the Filter phase calls the multi-dimensional resource profile plugin to filter nodes; the Score phase combines the multi-dimensional resource profile score and tenant weight to select the best node; the Permit phase executes the atomic commit / rollback logic of topology-aware GangScheduling.

[0205] The upstream of the extension point module 80 is connected to the scheduling cycle of the K8s scheduler, and the downstream of the extension point module 80 performs Pod binding and GPU resource allocation through the K8sAPI Server.

[0206] In the Kubernetes cluster of the intelligent computing center, multiple tenants share heterogeneous GPU resources (including different models of GPU cards, such as high-performance cards, mid-range cards, and domestically produced accelerator cards) to run distributed AI training tasks. Each training task consists of multiple workers, and the workers need to frequently communicate gradient synchronization, with communication efficiency directly affecting training speed. Different tenants have different SLA levels (such as high priority, medium priority, and low priority), and have differentiated requirements for resource guarantees and scheduling priorities. This invention builds a four-layer scheduling architecture on the Filter, Score, and Permit extension points of the K8s Scheduling Framework: "multi-dimensional resource profiling → joint decision-making of tenant quotas and priorities → topology-aware Gang Scheduling → time-division / space-division hybrid partitioning," to achieve three-dimensional joint optimization scheduling based on tenant SLA level, GPU topology affinity, and heterogeneous card type capability differences.

[0207] This invention is the first to integrate multiple dimensions for joint scoring decisions within the K8s scheduling framework, and achieves topology-aware Gang Scheduling and time-division / space-division hybrid partitioning, thereby maximizing distributed training communication efficiency and GPU cluster utilization while ensuring tenant SLA.

[0208] Example 2

[0209] Figure 3 This is a flowchart of the steps of a multi-tenant dynamic scheduling method for heterogeneous computing tasks in some other embodiments of the present invention.

[0210] like Figure 3As shown, in some embodiments, the multi-tenant dynamic scheduling method for heterogeneous computing tasks of the present invention includes the following steps:

[0211] Step S01: The scheduler receives the CRD (such as PyTorchJob) of the AI ​​training task and parses the required information such as the number of workers required for the task, the number of GPUs required for each worker, and card type preference.

[0212] Step S02: In the Filter stage, the multi-dimensional resource profiling plugin filters out nodes that do not meet the conditions according to the task requirements, including: card type mismatch, insufficient video memory, GPU utilization saturation, etc.

[0213] Step S03: In the Score stage, the multi-dimensional resource profiling plugin calculates a multi-dimensional profile score (including card type performance vector, memory fragmentation rate, computing power utilization, etc.) for each candidate node. At the same time, the tenant quota and priority joint decision module calculates the comprehensive scheduling weight of the tenant to which the current task belongs.

[0214] Optionally, the tenant's overall scheduling weight W k The calculation is performed using equation (1):

[0215] (1)

[0216] Where k represents the k-th tenant, These represent the SLA level weighting coefficient, usage weighting coefficient, and emergency weighting coefficient, respectively. Optional, , , . This represents the SLA rating score of tenant k. The numerical value can be mapped according to the tenant SLA level: High Priority = 1.0, Medium Priority = 0.7, Low Priority = 0.4. This represents the historical usage efficiency score of tenant k. The value is the ratio of the tenant's historical actual GPU utilization to the number of requests. This represents the urgency score of task i. The score is calculated based on the urgency of the deadline for task i relative to the current time; the closer the deadline, the higher the score.

[0217] Step S04: The topology-aware Gang Scheduling algorithm generates candidate worker node combinations, calculates the topology score (NVLink direct connection pairs and IB hop count) for each combination, and then scores it together with tenant weight and card type matching degree to obtain a comprehensive scheduling score.

[0218] Optionally, in some embodiments, topology score Calculate according to formula (2):

[0219] (2)

[0220] in, Represents node combinations Internal NVLink direct connection to GPU pairs, Represents node combinations The theoretical maximum value of the logarithm of the internal NVLink direct connection to the GPU. Represents the NVLink topology weight coefficient. Indicates the IB topology weight coefficient. Optional, , The more NVLink direct connections and the fewer the average number of IB hops, the higher the topology score. Represents node combinations Average number of IB hops between internal nodes. This indicates the maximum number of hops in the cluster's IB network.

[0221] Overall scheduling score Calculate according to formula (3):

[0222] (3)

[0223] in, Indicates task i in node combination The overall scheduling score. Represents node combinations The average card performance score of all nodes. Represents node combinations The average memory fragmentation rate of all nodes in the memory. This represents the average GPU computing power utilization of all nodes in the node combination 𝒩.

[0224] Step S05: Select the node combination with the highest overall score as the target worker node combination, and attempt to atomically bind all workers during the Permit phase. If any worker binding fails, roll back all combinations and regenerate candidate combinations. Continue until all workers are successfully bound.

[0225] The optimal node combination selection satisfies equation (4):

[0226] (4)

[0227] in, This is the set of all candidate worker node combinations that meet the task's resource requirements. The scheduler selects the node combination with the highest overall score. Perform atomic binding.

[0228] Step S06: After successful binding, the time-division / space-division hybrid partitioning engine automatically selects the GPU partitioning mode (exclusive / MIG space-division / vGPU time-division) based on the task profile, allocates GPU resources, and starts the AI ​​training task.

[0229] Example 3: Distributed Training Scheduling of Multi-tenant Heterogeneous GPU Clusters

[0230] Application Scenario: A Kubernetes cluster in a smart computing center contains various GPU nodes, including 8 high-performance GPU nodes (8 GPUs per node, fully interconnected via NVLink), 16 mid-range GPU nodes (4 GPUs per node, interconnected via PCIe), and 4 domestic accelerator GPU nodes (4 GPUs per node, interconnected via HCCL). The cluster is interconnected via an IB network, with 1 IB hop within the same rack and 2-3 IB hops across racks. Three tenants (high-priority, mid-priority, and low-priority) share this cluster to run distributed AI training tasks.

[0231] The scheduling process includes:

[0232] Step S10: The high-priority tenant submits an 8-worker PyTorchJob, with each worker requiring one high-performance card.

[0233] Step S20: The GPU node profiler collects metrics for each node in real time and updates the GPU Node Profile CRD.

[0234] Step S30: The multi-dimensional resource profiling plugin filters out mid-range card nodes and domestic acceleration card nodes (card type mismatch) in the Filter stage, and retains 8 high-performance card nodes.

[0235] Step S40: The tenant joint decision-making model calculates the weights of high-priority tenants.

[0236] .

[0237] Step S50: The topology-aware Gang Scheduling algorithm generates candidate worker node combinations, prioritizing 8-card nodes with full NVLink interconnection within the same rack (those with the highest topology score).

[0238] Step S60: After calculating the overall scheduling score, select the optimal node combination and atomically bind 8 workers during the Permit phase.

[0239] Step S70: Due to the high estimated GPU utilization of the task (>0.5), the time-division / space-division hybrid partitioning engine allocates a dedicated GPU to each worker.

[0240] Technical Results: All 8 workers are scheduled to the same high-performance card node. Gradient synchronization between workers is achieved via direct NVLink connection, with a communication bandwidth of 600GB / s, approximately 50 times higher than cross-rack IB communication (bandwidth approximately 100Gbps). End-to-end training task time is reduced by approximately 40%. Parameter Setting Example: , , , , , , Second, .

[0241] Example 4: vGPU time-division sharing and mixed scheduling for low-priority tasks

[0242] Application scenario: In the same cluster, a low-priority tenant submits multiple lightweight inference tasks, each of which requires only a small amount of GPU computing power, and the estimated GPU utilization is less than 0.3.

[0243] The scheduling process specifically includes:

[0244] Step S10: Low-priority tenants submit multiple single-GPU inference tasks.

[0245] Step S20: The tenant joint decision-making model calculates the weights of low-performing tenants.

[0246] .

[0247] Step S30: The multi-dimensional resource profiling plugin identifies that some high-performance card nodes already have MIG instances running, but still have available vGPU time-sharing resources.

[0248] Step S40: The time-division / space-division hybrid partitioning engine determines that the estimated GPU utilization of the task is lower than uthresh=0.5, and selects the vGPU time-division sharing mode.

[0249] Step S50: Multiple inference tasks share the same physical GPU time slice in vGPU mode, coexisting with the existing MIG instance on the node.

[0250] Technical Effects: Previously, in dedicated mode, a single GPU could only run one task. With vGPU time-sharing, 3-5 low-utilization inference tasks can run simultaneously, increasing single-GPU utilization from approximately 30% to over 80%. Furthermore, MIG instances and vGPU instances coexist on the same node, maximizing resource utilization in mixed deployment scenarios. Parameter Setting Example: , The default vGPU time slice is 10ms, and the maximum number of vGPU instances per card is 7.

[0251] Example 5: Gang Scheduling Atomic Rollback and Rescheduling

[0252] Application scenario: A tenant in the middleware area submits a 4-worker distributed training task, which requires 4 middleware cards.

[0253] The scheduling process specifically includes:

[0254] Step S10: The topology-aware Gang Scheduling algorithm generates candidate worker node combinations, with the first choice being 4 mid-level card nodes within the same rack (IB hop count = 1).

[0255] Step S20: In the Permit phase, attempt to atomically bind 4 workers. 3 of them are successfully bound, and 1 fails to bind because the node resources are preempted by a high-priority task.

[0256] Step S30: Gang Scheduling detects incomplete binding, triggers a full rollback, and releases the 3 bound worker resources.

[0257] Step S40: The algorithm regenerates candidate combinations, selects another group of 4 mid-range card nodes (across racks, IB hop count = 2), and re-attempts atomic binding.

[0258] All four workers were successfully bound, and the task was started.

[0259] Technical benefits: The atomic rollback mechanism avoids resource deadlock caused by three workers occupying resources while a fourth cannot be scheduled. Rescheduling after rollback ensures the task eventually starts successfully, while prioritizing combinations with better topologies, thus maintaining training efficiency even in resource-contention scenarios.

[0260] The present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the multi-tenant dynamic scheduling method for heterogeneous computing tasks described in any of the above embodiments.

[0261] In summary, compared with the prior art, the present invention has the following advantages:

[0262] (1) Three-dimensional joint optimization, globally optimal scheduling decision: This invention is the first to integrate three dimensions—tenant SLA level, GPU topology affinity, and heterogeneous card capability differences—into a K8s scheduling framework for joint scoring and decision-making (Equation 3), avoiding the overall suboptimal problem caused by single-dimensional or dual-dimensional optimization in existing technologies. Through comprehensive scheduling scores... The multiplicative coupling structure ensures that the highest score is obtained only when all three dimensions are satisfied simultaneously.

[0263] (2) Cluster-level topology-aware Gang Scheduling with minimized communication overhead: The topology-aware GangScheduling algorithm of this invention searches for the node combination that maximizes the number of NVLink direct connections and minimizes the number of IB hops within the cluster (Equations 2 and 4), rather than simply co-locating nodes within the same topology domain. Compared with existing node-level or domain-level topology-aware schemes, it can significantly reduce the proportion of cross-rack communication during distributed training.

[0264] (3) Time-division / space-division hybrid partitioning significantly improves GPU utilization: This invention makes unified decisions on the MIG space-division and vGPU time-division partitioning modes of the GPU at the scheduling layer, automatically selects the optimal partitioning strategy based on the estimated GPU utilization of the task, and supports the coexistence of MIG instances and vGPU instances on the same node. Compared with existing solutions that only support a single mode, this invention can significantly improve the overall GPU utilization.

[0265] (4) Real-time multi-dimensional resource profile, with strong timeliness of scheduling decision: This invention generates a multi-dimensional profile in real time, including card type performance vector, NVLink topology matrix, memory fragmentation rate and computing power utilization rate, through a custom GPU node profile collector. Compared with the existing schemes based on static labels or coarse-grained indicators, the scheduling decision is more timely and accurate.

[0266] (5) Atomicity scheduling guarantee to avoid resource deadlock: This invention implements the atomic commit / rollback mechanism of Gang Scheduling through the Permit extension point of the K8s Scheduling Framework, ensuring that all workers of the distributed training task are either all scheduled successfully or all rolled back, thus avoiding resource occupation and deadlock problems caused by partial scheduling.

[0267] Figure 4 This is a schematic diagram of the composition structure of a computer device in some embodiments of the present invention.

[0268] For example, such as Figure 4 As shown, an embodiment of the present invention provides a computer device comprising:

[0269] One or more processors 901;

[0270] The memory 902 stores one or more programs, which, when executed by one or more processors, enable the one or more processors to implement the multi-tenant dynamic scheduling method for heterogeneous computing tasks described above.

[0271] One or more I / O interfaces 903 are connected between the processor and memory and configured to enable data interaction between the processor and memory.

[0272] Among them, processor 901 is a device with data processing capabilities, including but not limited to central processing unit (CPU); memory 902 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory (FLASH); I / O interface (read-write interface) 903 is connected between processor 901 and memory 902, enabling data interaction between processor 901 and memory 902, including but not limited to data bus (Bus).

[0273] In some embodiments, the processor 901, memory 902, and I / O interface 903 are interconnected via a bus, and thus connected to other components of the computing device.

[0274] In other embodiments, the present invention provides a computer-readable medium. The computer-readable medium stores a computer program, wherein, when executed by a processor, the program implements the multi-tenant dynamic scheduling method for heterogeneous computing tasks provided in any embodiment of the present invention.

[0275] It will be understood by those skilled in the art that all or some of the steps, modules / units in the methods, systems, and apparatuses disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. In hardware implementations, the division between modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0276] It is understood that the above embodiments are merely exemplary implementations used to illustrate the principles of the present invention, and the present invention is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and essence of the present invention, and these modifications and improvements are also considered to be within the scope of protection of the present invention.

Claims

1. A multi-tenant dynamic scheduling method for heterogeneous computing tasks, applied to a distributed computing cluster, wherein the distributed computing cluster includes multiple worker nodes, characterized in that, include: Obtain heterogeneous computing tasks, parse the heterogeneous computing tasks to obtain task requirements, and determine the tenant to which the heterogeneous computing tasks belong and the multi-dimensional tenant attributes corresponding to the tenant. Obtain multi-dimensional indicator data of the worker nodes in the distributed computing cluster, and filter out multiple candidate worker nodes based on the task requirements and the multi-dimensional indicator data of the worker nodes. Calculate a multidimensional resource profile for each candidate working node based on the multidimensional indicator data of the candidate working nodes, and generate at least one combination of candidate working nodes based on the multidimensional resource profiles. The comprehensive scheduling weight of the tenant to which the heterogeneous computing task belongs is calculated based on the multidimensional tenant attributes; For each candidate work node combination, calculate the corresponding topology score, and combine it with the comprehensive scheduling weight and the card type matching degree between the task requirements and the candidate work node combination to obtain the comprehensive scheduling score; Based on the comprehensive scheduling score, a target working node combination is determined from the at least one group of candidate working node combinations, and the heterogeneous computing task is scheduled to be executed in the target working node combination.

2. The multi-tenant dynamic scheduling method for heterogeneous computing tasks according to claim 1, characterized in that, The working node is equipped with a heterogeneous acceleration card; The heterogeneous accelerator card includes at least one of a graphics processing unit, a neural network processor, and a field-programmable gate array; The multidimensional indicator data includes at least one of the following: accelerator card model, video memory capacity, node internal interconnection link status, inter-node communication network topology, accelerator card computing unit utilization rate, and allocated video memory.

3. The multi-tenant dynamic scheduling method for heterogeneous computing tasks according to claim 2, characterized in that, The multi-dimensional resource profile includes: card type performance vector, memory fragmentation rate, topology matrix, and current computing power utilization. The calculation of the multidimensional resource profile based on the multidimensional indicator data specifically includes: Determine the card type performance vector based on the accelerator card model; The video memory fragmentation rate is calculated based on the video memory capacity and the amount of video memory already allocated. Construct a topology matrix based on the internal interconnection link status of the nodes; The current computing power utilization rate is evaluated based on the occupancy rate of the accelerator card's computing units.

4. The multi-tenant dynamic scheduling method for heterogeneous computing tasks according to claim 1, characterized in that, The multidimensional tenant attributes include at least one of the following: SLA level, resource quota limit, historical usage efficiency statistics, and task urgency label; The comprehensive scheduling weight of the tenant to which the heterogeneous computing task belongs is calculated based on the multidimensional tenant attributes, including: the comprehensive scheduling weight is obtained by weighting the SLA level, the historical usage efficiency statistics and the task urgency label.

5. The multi-tenant dynamic scheduling method for heterogeneous computing tasks according to any one of claims 1 to 4, characterized in that, Based on the multi-dimensional resource profile, at least one set of candidate working node combinations is generated, specifically including: Based on the multi-dimensional resource profile, candidate working nodes that meet preset conditions are selected from the multiple working nodes; Based on the group scheduling strategy, a node combination that meets the requirements of the number of working nodes and card type for the heterogeneous computing task is selected from the candidate working nodes. Obtain the physical communication link status between each working node in the distributed computing cluster, parse the cluster communication topology graph, and calculate the topology connectivity score within each node combination based on the cluster communication topology graph. The node combinations whose topological connectivity scores meet a preset threshold are determined as the candidate working node combinations.

6. The multi-tenant dynamic scheduling method for heterogeneous computing tasks according to any one of claims 1 to 4, characterized in that, Before scheduling the heterogeneous computing task to the target worker node combination, the method further includes: attempting to atomically bind all worker nodes in the target worker node combination; if the binding of any worker node in the target worker node combination fails, performing a rollback operation on the already bound worker nodes and regenerating the target worker node combination until all worker nodes in the target worker node combination are successfully bound.

7. The multi-tenant dynamic scheduling method for heterogeneous computing tasks according to claim 6, characterized in that, After all worker nodes in the target worker node combination are successfully bound, the method further includes: The task profile of the heterogeneous computing task is analyzed, and the task profile includes GPU utilization estimation, memory requirements and memory isolation identifier. Based on the task profile, the target GPU partitioning mode is determined from the dedicated GPU mode, the multi-instance GPU space-sharing mode, and the virtual GPU time-sharing mode. Allocate GPU resources to the heterogeneous computing task according to the target GPU partitioning mode, and start the heterogeneous computing task.

8. The multi-tenant dynamic scheduling method for heterogeneous computing tasks according to claim 7, characterized in that, Based on the task profile, the target GPU partitioning mode is determined from the following: dedicated GPU mode, multi-instance GPU space-sharing mode, and virtual GPU time-sharing mode, including: When the estimated GPU utilization rate is higher than the preset utilization rate threshold, or when the memory isolation flag indicates that strong isolation is required, the heterogeneous computing task is allocated an exclusive GPU mode or a MIG space division mode. When the estimated GPU utilization is lower than the preset utilization threshold and the memory isolation flag indicates that sharing is allowed, vGPU time-sharing mode is allocated to the heterogeneous computing task.

9. The multi-tenant dynamic scheduling method for heterogeneous computing tasks according to claim 7, characterized in that, Allocating GPU resources to the heterogeneous computing task according to the target GPU partitioning mode includes: The node device plugin reports to the distributed computing cluster the allocable GPU resource instances and quantities exposed by each worker node in the target worker node combination under the target GPU partitioning mode. On the same worker node in the target worker node combination, MIG space-division mode and vGPU time-division sharing mode are run respectively to support the hybrid deployment of heterogeneous GPU resources.

10. A multi-tenant dynamic scheduling system for heterogeneous computing tasks, applied to a distributed computing cluster, the distributed computing cluster comprising multiple worker nodes, characterized in that... The system includes: The task analysis module is used to acquire heterogeneous computing tasks and parse the heterogeneous computing tasks to obtain task requirements. The tenant quota and SLA management module is used to determine the tenant to which the heterogeneous computing task belongs and the multi-dimensional tenant attributes corresponding to the tenant based on the heterogeneous computing task. The multi-dimensional resource profiling plugin module is used to obtain multi-dimensional indicator data of the working nodes in the distributed computing cluster, filter out multiple candidate working nodes according to the task requirements and the multi-dimensional indicator data of the working nodes, and calculate the multi-dimensional resource profile of each candidate working node according to the multi-dimensional indicator data of the candidate working nodes. The tenant quota and priority joint decision-making module is used to calculate the comprehensive scheduling weight of the tenant to which the heterogeneous computing task belongs based on the multi-dimensional tenant attributes. The topology-aware module is used to generate at least one set of candidate worker node combinations based on the multi-dimensional resource profile, calculate the corresponding topology score for each candidate worker node combination, and obtain a comprehensive scheduling score by combining the comprehensive scheduling weight and the card type matching degree between the task requirements and the candidate worker node combinations; based on the comprehensive scheduling score, determine the target worker node combination from the at least one set of candidate worker node combinations, and schedule the heterogeneous computing task to the target worker node combination for execution.

11. The multi-tenant dynamic scheduling system for heterogeneous computing tasks according to claim 10, characterized in that, The system also includes: The worker node profile acquisition module is used to collect initial indicator data of worker nodes in the distributed computing cluster. The cluster status monitoring module is used to monitor the running status of the worker nodes in the distributed computing cluster. The multi-dimensional resource profiling plugin module is used to obtain multi-dimensional indicator data of the worker nodes in the distributed computing cluster based on the initial indicator data collected by the worker node profiling acquisition module and the data monitored by the cluster status monitoring module.

12. The multi-tenant dynamic scheduling system for heterogeneous computing tasks according to claim 10, characterized in that, The system further includes: an allocation engine module, used to parse the task profile of the heterogeneous computing task, the task profile including GPU utilization estimation, memory requirements and memory isolation identifier; based on the task profile, determine the target GPU partitioning mode from the exclusive GPU mode, multi-instance GPU space partitioning mode and virtual GPU time partitioning mode; allocate GPU resources to the heterogeneous computing task according to the target GPU partitioning mode, and start the heterogeneous computing task.

13. The multi-tenant dynamic scheduling system for heterogeneous computing tasks according to claim 10, characterized in that, The topology-aware module is further configured to attempt to atomically bind all worker nodes in the target worker node combination before scheduling the heterogeneous computing task to the target worker node combination. If the binding of any worker node in the target worker node combination fails, a rollback operation is performed on the already bound worker nodes, and the target worker node combination is regenerated until all worker nodes in the target worker node combination are successfully bound.

14. The multi-tenant dynamic scheduling system for heterogeneous computing tasks according to any one of claims 10 to 13, characterized in that, The scheduling framework of the distributed computing cluster is configured with a custom scheduling plugin, which includes: Filter extension point, Score extension point and Permit extension point; The Filter extension point is used to call the multi-dimensional resource profiling plugin module to filter out multiple candidate working nodes that meet the resource requirements of heterogeneous computing tasks. The Score extension point is used to call the multi-dimensional resource profiling plugin module to calculate the multi-dimensional resource profile of the candidate worker nodes, and combine it with the comprehensive scheduling weight of the tenants in the task calculated by the tenant quota and priority joint decision module and the topology score calculated by the topology awareness module to obtain the comprehensive score of the candidate worker nodes, thereby determining the target worker node combination. The Permit extension point is used to call the topology-aware module to perform atomic binding on all worker nodes in the target worker node combination. If the binding of any worker node fails, a rollback operation is performed on the reserved worker nodes. After the atomic binding is successful, the target worker node combination is approved to enter the binding cycle so that the heterogeneous computing tasks can be bound to the target worker node combination for execution.

15. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the multi-tenant dynamic scheduling method for heterogeneous computing tasks as described in any one of claims 1 to 9.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the multi-tenant dynamic scheduling method for heterogeneous computing tasks as described in any one of claims 1 to 9.

Images