Heterogeneous AI computing power resource scheduling method and system

Abstract

The invention discloses a heterogeneous AI computing power resource scheduling method and system, and the method comprises the steps: constructing a heterogeneous AI computing power resource pool, wherein the heterogeneous AI computing power resource pool integrates the computing resources of a plurality of heterogeneous AI acceleration chips; obtaining a scheduling demand of the AI task, wherein the scheduling demand comprises a task type, a resource request quantity, a priority identifier and a task group association relationship; generating a multi-dimensional scheduling strategy according to task requirements, wherein the scheduling strategy comprises a priority scheduling rule, an affinity scheduling rule and a resource preemption rule; based on a multi-dimensional scheduling strategy, the AI tasks are dynamically allocated to target computing power nodes of the heterogeneous AI computing power resource pool, and the task execution state and the resource utilization rate are monitored in real time; and dynamically adjusting computing resource allocation according to the resource utilization rate. Through the heterogeneous AI computing power resource pool, the resource utilization rate is remarkably improved, dynamic resource allocation is realized through a multi-dimensional scheduling strategy, and meanwhile, a communication path is optimized through an affinity scheduling strategy, so that the problem of task starvation caused by resource fragmentation is avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of computing power resource scheduling, and in particular to a heterogeneous AI computing power resource scheduling method and system. Background Art

[0002] With the rapid development of artificial intelligence technology, large-scale model training and inference tasks are placing higher demands on computing infrastructure. Traditional AI computing power scheduling methods are mainly designed for computing resources with a single architecture (such as NVIDIA GPUs), and are unable to effectively cope with the complex scenarios in which multiple types of acceleration chips (such as GPUs, NPUs, and DCUs) coexist in today's heterogeneous computing environments. In actual applications, due to significant differences in hardware architectures between different manufacturers, their driver interfaces, resource management mechanisms, and performance characteristics vary, leading to increasingly prominent problems such as severe resource fragmentation and low utilization. For example, in some scenarios, the disordered scheduling of small-scale tasks results in the remaining node resources being unable to meet the needs of large-scale tasks, and traditional coarse-grained resource allocation methods further exacerbate resource waste, resulting in high hardware procurement costs. In addition, facing the differentiated task priority requirements of key industries such as finance and healthcare, existing scheduling strategies cannot achieve dynamic preemption of low-priority resources by high-priority tasks, resulting in increased latency in critical business responses. While existing technologies attempt to optimize resource management through containerized deployment and virtualized partitioning, they still lack a comprehensive heterogeneous computing power scheduling system. In particular, they face significant deficiencies in unified resource abstraction, dynamic priority adjustment, and multi-layer network collaboration. These issues severely restrict the flexibility and reliability of AI computing power infrastructure in complex business scenarios. There is an urgent need for a heterogeneous AI computing power scheduling method that is compatible with multiple types of accelerator chips, supports fine-grained resource partitioning, and implements dynamic resource optimization based on multi-dimensional scheduling strategies. Summary of the Invention

[0003] In view of this, the present invention proposes a heterogeneous AI computing resource scheduling method and system, which can realize the dynamic scheduling of computing resources. The present invention provides the following technical solutions:

[0004] A method for scheduling heterogeneous AI computing resources, the method comprising:

[0005] Building a heterogeneous AI computing power resource pool that integrates computing resources from multiple heterogeneous AI acceleration chips;

[0006] Obtaining the scheduling requirements of the AI ​​task, including the task type, resource request amount, priority identifier, and task group association;

[0007] Generate a multi-dimensional scheduling strategy based on the task requirements, wherein the scheduling strategy includes priority scheduling rules, affinity scheduling rules, and resource preemption rules;

[0008] Based on the multi-dimensional scheduling strategy, AI tasks are dynamically allocated to target computing nodes in the heterogeneous AI computing resource pool, and task execution status and resource utilization are monitored in real time;

[0009] The computing resource allocation is dynamically adjusted according to the resource utilization.

[0010] Optionally, the building of a heterogeneous AI computing resource pool, wherein the heterogeneous AI computing resource pool integrates computing resources of multiple heterogeneous AI acceleration chips, includes:

[0011] Fine-grained segmentation of the computing cores and memory resources of multiple heterogeneous AI acceleration chips, including at least GPUs, NPUs, and DCUs, is performed. The GPU is segmented using multi-instance GPU-based segmentation technology to divide single-card resources into multiple independent instances; the NPU is segmented using virtualized NPU technology to divide single-card resources into multiple virtual instances; and the DCU is segmented using virtualized DCU technology to divide single-card resources into multiple virtual instances.

[0012] Abstract and encapsulate the segmented heterogeneous AI acceleration chip resources, shielding the underlying hardware differences and forming standardized resource units;

[0013] A heterogeneous AI computing power resource pool is constructed based on the standardized resource units.

[0014] Optionally, obtaining the scheduling requirements of the AI ​​task, wherein the scheduling requirements include the task type, resource request amount, priority identifier, and task group association, includes:

[0015] Obtain the task type of the AI ​​task through the task description file or API interface. The task types include: training task, inference task, and edge computing task;

[0016] Obtain resource requests using resource profiling technology, including video memory requirements, computing core requirements, and communication loan requirements;

[0017] The priority identifier is generated by user configuration or system automatically, and the priority identifier includes: user-defined priority and dynamic priority adjustment;

[0018] Obtain task group association relationships through task dependency analysis.

[0019] Optionally, the multi-dimensional scheduling strategy is generated according to the task requirements, and the scheduling strategy includes priority scheduling rules, affinity scheduling rules, and resource preemption rules, including:

[0020] generating a priority scheduling rule based on the task priority identifier;

[0021] generating affinity scheduling rules based on the task group association relationship;

[0022] Generate resource preemption rules based on the resource process status.

[0023] Optionally, dynamically allocating AI tasks to target computing nodes of the heterogeneous AI computing resource pool based on the multi-dimensional scheduling strategy, and monitoring task execution status and resource utilization in real time includes:

[0024] Dynamically assign AI tasks to target computing nodes through a distributed scheduling engine based on the multi-dimensional scheduling strategy;

[0025] The unified monitoring interface layer collects task execution status and resource utilization indicators in real time.

[0026] Optionally, dynamically adjusting the computing resource allocation according to the resource utilization includes:

[0027] Generate dynamic resource utilization adjustment strategies based on hardware, task, and network layer monitoring data collected by the unified monitoring interface layer, including resource recycling trigger conditions, fragmented resource merging rules, and priority dynamic adjustment models;

[0028] Resource adjustment operations are performed through the distributed scheduling engine, including resource recovery and reallocation and task priority rebalancing.

[0029] The present invention further discloses a heterogeneous AI computing resource scheduling system, comprising:

[0030] A resource construction module is used to build a heterogeneous AI computing power resource pool that integrates the computing resources of multiple heterogeneous AI acceleration chips;

[0031] A demand acquisition module is used to obtain the scheduling requirements of AI tasks, which include task type, resource request amount, priority identifier, and task group association relationship;

[0032] A scheduling strategy generation module, comprising generating a multi-dimensional scheduling strategy according to the task requirements, wherein the scheduling strategy includes priority scheduling rules, affinity scheduling rules and resource preemption rules;

[0033] A task allocation module, configured to dynamically allocate AI tasks to target computing nodes in the heterogeneous AI computing resource pool based on the multi-dimensional scheduling strategy, and monitor task execution status and resource utilization in real time;

[0034] An allocation adjustment module is used to dynamically adjust the computing resource allocation according to the resource utilization.

[0035] The present invention further discloses a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program implements the above method when executed by a processor.

[0036] The present invention further discloses an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above method when executing the program.

[0037] The present invention further discloses a computer program product, comprising a computer program, which implements the above method when executed by a processor.

[0038] According to the technical solution of the present invention, by constructing a heterogeneous AI computing power resource pool and combining hardware virtualization segmentation technology to achieve fine-grained division of computing cores and video memory, resource utilization is significantly improved. In the task scheduling stage, dynamic resource allocation is achieved through a multi-dimensional scheduling strategy: priority scheduling rules generated based on task type and resource request amount, combined with user-defined priority and dynamic priority adjustment model, ensure the ability of high-priority tasks to preempt low-priority resources, shortening the response of key businesses; at the same time, the affinity scheduling strategy designed for the association relationship of task groups optimizes the communication path through node grouping binding and network topology perception mechanism, avoiding the problem of task starvation caused by resource fragmentation. During the task execution process, through distributed real-time data collection, resource recovery, fragmented resource merging and priority rebalancing are dynamically triggered, thereby improving resource utilization; in summary, the present invention constructs a full-link heterogeneous computing power management closed loop covering resource pooling, task scheduling, and dynamic adjustment, providing the application industry with a highly reliable, low-latency, and high-resource-utilization AI computing power infrastructure solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] For purposes of illustration and not limitation, the present invention will now be described with reference to embodiments thereof and the accompanying drawings, in which:

[0040] Figure 1 Schematic diagram of a heterogeneous AI computing resource scheduling method in an embodiment of the present invention;

[0041] Figure 2 Schematic diagram of the components of the heterogeneous AI computing resource scheduling system in an embodiment of the present invention;

[0042] Figure 3 Schematic diagram of the structure of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0043] In order to enable those skilled in the art to better understand the solution of this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of this application.

[0044] It should be noted that, in the absence of conflict, the embodiments of the present application and the features thereof can be combined with each other. The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0045] refer to Figure 1 This embodiment discloses a method for scheduling heterogeneous AI computing resources, which includes the following steps:

[0046] S100: Build a heterogeneous AI computing power resource pool, which integrates computing resources of multiple heterogeneous AI acceleration chips.

[0047] Specifically, the computing cores and video memory resources of multiple heterogeneous AI acceleration chips, including at least GPU, NPU and DCU, are fine-grainedly segmented, wherein: the GPU is segmented based on multi-instance GPU to divide the resources of a single card into multiple independent instances; the NPU is segmented by virtualized NPU technology to divide the resources of a single card into multiple virtual instances; the DCU is segmented by virtualized DCU technology to divide the resources of a single card into multiple virtual instances; the segmented heterogeneous AI acceleration chip resources are abstracted and encapsulated to shield the underlying hardware differences and form standardized resource units; and a heterogeneous AI computing power resource pool is constructed based on the standardized resource units.

[0048] In this embodiment, for example, by integrating the physical resources of heterogeneous AI acceleration chips from multiple manufacturers, such as NVIDIA GPU, Huawei Ascend NPU, and Hygon DCU, hardware-level virtualization technologies such as NVIDIA MIG, Ascend vNPU, and Hygon vDCU are used to perform fine-grained segmentation of the computing core and memory resources of each chip. For example, the NVIDIA A100 GPU can be divided into up to 7 independent instances, with a minimum granularity of 1 SM (computing core) + 5GB of video memory; the Ascend 910 NPU supports virtualization segmentation through the CANN Toolkit, with a minimum granularity of 2 AI Cores + 1GB of memory; the Hygon DCU Z100L is divided into 4 instances based on the DTK tool, with a minimum granularity of 1 CU + 1MB of video memory. The segmented resources are abstracted and encapsulated through the unified resource management interface of the digital native engine EOS and the Panshi cloud operating system ESCL to form standardized resource units. This interface layer shields the underlying hardware differences and enables unified management of cross-architecture (x86 / ARM / c86) resources. For example, the vNPU instance of the Ascend NPU and the MIG instance of the NVIDIA GPU are mapped to virtualized resource tags of the same granularity for identification by the upper-layer scheduler. Based on this, the constructed heterogeneous AI computing resource pool supports dynamic expansion, is compatible with the seamless access of future new AI chips, and realizes hybrid scheduling of multiple instruction sets (such as x86+NVIDIA, Kunpeng+Ascend, and Hygon+DCU) through a one-cloud, multi-core architecture. For example, in a certain bank's production cloud scenario, the resource pool successfully managed more than 4,000 trusted computing servers, integrating a hybrid architecture of Kunpeng CPU+Ascend NPU and Hygon CPU+DCU, and realizing cross-vendor resource scheduling for financial risk control model training tasks through unified resource tags, which increased resource utilization by more than 50% and reduced hardware procurement costs by 700%.

[0049] Step S100 not only solves the resource fragmentation problem caused by the traditional whole-card monopoly, such as the edge inference scenario only requires 10% computing power but occupies the entire card, but also avoids manufacturer binding through a standardized abstraction layer, and provides underlying support for subsequent multi-dimensional scheduling strategies (priority scheduling, affinity scheduling), forming a complete technical closed loop from resource pooling to dynamic adjustment.

[0050] S200: Obtain the scheduling requirements of the AI ​​task, which include the task type, resource request amount, priority identifier, and task group association. The task type of the AI ​​task is obtained through a task description file or API interface, and the task types include: training tasks, inference tasks, and edge computing tasks; the resource request amount is obtained through resource profiling technology, and the resource request amount includes: graphics memory requirements, computing core requirements, and communication loan requirements; the priority identifier is generated through user configuration or system automatic generation, and the priority identifier includes: user-defined priority and dynamic priority adjustment; and the task group association is obtained through task dependency analysis.

[0051] In this embodiment, the task type label of the AI ​​task is automatically parsed through the task description file or API interface, and the resource request amount is dynamically estimated in combination with the resource portrait technology. At the same time, the priority identifier is determined by the dual mode of user configuration and system automatic generation, and finally the task group association relationship is constructed through task dependency analysis to form a complete scheduling demand input. In the specific implementation process, the taskType field in the task description file is first identified. This field contains the identification of training tasks, inference tasks, or edge computing tasks. For example, the financial risk control model training task will be marked as training type and trigger the distributed training resource configuration process. The task model parameter scale is then analyzed through the resource portrait engine to dynamically estimate the video memory requirements (such as BERT-base requires 8GB of video memory, GPT-3 requires 700GB of video memory), computing core requirements, and communication bandwidth requirements.

[0052] S300: Generate a multi-dimensional scheduling strategy based on the task requirements. The scheduling strategy includes priority scheduling rules, affinity scheduling rules, and resource preemption rules. Specifically, the priority scheduling rule is generated based on the task priority identifier; the affinity scheduling rule is generated based on the task group association relationship; and the resource preemption rule is generated based on the resource process status.

[0053] By integrating multi-dimensional data on task priority identification, task group relationships, and resource supply and demand status, priority scheduling rules, affinity scheduling rules, and resource preemption rules are dynamically constructed to form an intelligent scheduling system covering the entire task lifecycle. During implementation, the generation of priority scheduling rules is based on a dual-mode trigger mechanism based on task priority identification: users manually set three levels of priority (high / medium / low), and the system dynamically adjusts the priority based on task waiting time and SLA compliance rate. For example, when the waiting time for a credit approval task exceeds 30 minutes, its priority weight is automatically increased by 20%, triggering the Volcano scheduling engine to reorder and ensure that high-priority tasks have priority access to Ascend NPU clusters or GPU NVLink interconnected node group resources. The generation of affinity scheduling rules relies on in-depth analysis of task group relationships, and node group affinity and anti-affinity strategies are automatically generated through a labeling system. For example, a smart investment research task in the securities industry includes a data preprocessing container and a BERT training container. The system automatically identifies it as a collaborative training task group and, combined with the AllReduce topology, binds the tasks to the same Ascend NPU cluster. It allocates two vir04 instances for the preprocessing phase and four vir08 instances for the training phase. At the same time, privacy-sensitive tasks and non-sensitive tasks are physically isolated and deployed to avoid resource conflicts. Resource preemption rules are generated based on real-time data on resource supply and demand, combined with fragmented resource merging rules and dynamic priority models to achieve flexible adjustment. When cluster resources are insufficient, high-priority tasks can preempt the reserved resources of low-priority tasks, but the remaining resources after preemption must still meet the minimum requirements of low-priority tasks (such as 1 SM core + 5GB of video memory). For example, in a bank's ICT production cloud scenario, a hybrid Kunpeng CPU and Ascend NPU architecture server automatically releases fragmented resources based on resource recycling triggers (e.g., Ascend AI Core utilization <15% for 5 minutes). Combined with the scheduling engine's compact scheduling algorithm, it migrates small tasks to other nodes, increasing resource utilization to 82% and reducing task starvation by 95%. This technical solution reduces the response latency of financial risk control tasks by 30% through a dynamic priority adjustment mechanism. Combining affinity scheduling with communication topology optimization reduces the AllReduce parameter synchronization time for BERT pre-training tasks by 40%. Furthermore, resource recycling and rebalancing strategies significantly reduce hardware procurement costs.

[0054] S400: Based on the multi-dimensional scheduling strategy, dynamically allocate AI tasks to target computing nodes of the heterogeneous AI computing resource pool, and monitor task execution status and resource utilization in real time.

[0055] Through the distributed scheduling engine, combined with task priority identification, resource supply and demand status, and communication topology perception capabilities, AI tasks are accurately matched to target nodes in the heterogeneous computing resource pool, and full-link status tracking and dynamic optimization of resource utilization are achieved through a unified monitoring interface layer. Specifically, the scheduling engine first parses the priorityClassName field in the task description file, and generates a priority sorting strategy based on the dynamic priority adjustment model. In terms of affinity scheduling, through the analysis results of the association between the AllReduce topology and the task group, the collaborative training task group is bound to the same Ascend NPU cluster, and the inference service chain task group is allocated to the RoCE parameter network low-latency node according to the communication bandwidth requirements. Resource preemption rules are dynamically triggered based on real-time monitoring data: when cluster resources are insufficient, high-priority tasks can preempt the reserved resources of low-priority tasks, but the remaining resources must still guarantee the minimum requirements of low-priority tasks.

[0056] In terms of task execution status and resource utilization monitoring, NVIDIADCGMExporter, Ascend NPUExporter, Hygon DCUEXporter and other multi-vendor components are integrated through a unified monitoring interface layer to collect hardware layer, task layer and network layer indicators in real time.

[0057] S500: Dynamically adjust the computing resource allocation according to the resource utilization.

[0058] Through the unified monitoring interface layer, multi-dimensional indicators of the hardware layer, task layer, and network layer are collected in real time. Combined with the dynamic priority adjustment model and resource recovery trigger conditions, the distributed scheduling engine is driven to perform resource recovery, fragment merging, and priority rebalancing operations, forming a closed-loop optimization mechanism from monitoring to scheduling. Specifically, hardware layer data, including Ascend NPU temperature, NVIDIA GPU memory usage, and Hygon DCU memory usage, are obtained through components such as NVIDIA DCGM Exporter, Ascend NPU Expresser, and Hygon DCUE Expresser; task layer data, including BERT pre-training task progress, GPU utilization, and NPU AI Core utilization, is collected through the Kubernetes API; and communication bottleneck nodes are identified through RoCE parameter network bandwidth monitoring. Based on the above data, a dynamic adjustment strategy is generated. This implementation provides the following example: When Ascend AI Core utilization consistently falls below 15% or NVIDIA GPU utilization falls below 10%, a resource reclamation mechanism is triggered. The scheduling engine evicts low-priority task container instances and reinjects the released vir08 virtual instances into the resource pool. To address resource fragmentation, a compact scheduling algorithm is used to migrate small tasks to other nodes, consolidating fragmented resources to meet the needs of larger tasks. The dynamic priority adjustment model calculates task priority using an exponentially weighted moving average algorithm. For example, if the wait time for a credit approval task exceeds 30 minutes, its priority is automatically increased by 20%, triggering reordering and preempting lower-priority resources. During the resource adjustment execution phase, an AllReduce topology is automatically generated to optimize the parameter synchronization path for distributed training tasks, leveraging the low-latency RoCE parameter network to reduce cross-node communication time. If an Ascend NPU temperature exceeds the limit or an NVIDIA GPU memory error is detected, the BERT fine-tuning task is automatically migrated to a healthy node. Training snapshots are saved and restored using Kubernetes PersistentVolumes to ensure task continuity.

[0059] refer to Figure 2 This embodiment further discloses a heterogeneous AI computing resource scheduling system, including:

[0060] The resource construction module 21 is used to build a heterogeneous AI computing power resource pool, which integrates the computing resources of multiple heterogeneous AI acceleration chips, including: fine-grained segmentation of the computing cores and video memory resources of multiple heterogeneous AI acceleration chips including at least GPU, NPU and DCU, wherein: the GPU is divided into multiple independent instances by a segmentation technology based on multi-instance GPU; the NPU is divided into multiple virtual instances by a virtualized NPU technology; the DCU is divided into multiple virtual instances by a virtualized DCU technology; the segmented heterogeneous AI acceleration chip resources are abstracted and encapsulated to shield the underlying hardware differences and form standardized resource units; and a heterogeneous AI computing power resource pool is constructed based on the standardized resource units.

[0061] The demand acquisition module 22 is used to obtain the scheduling requirements of AI tasks, and the scheduling requirements include task type, resource request amount, priority identifier and task group association relationship, including: obtaining the task type of AI tasks through task description files or API interfaces, and the task types include: training tasks, inference tasks, and edge computing tasks; obtaining resource request amounts through resource profiling technology, and the resource request amounts include: video memory requirements, computing core requirements and communication loan requirements; generating priority identifiers through user configuration or system automatic generation, and the priority identifiers include: user-defined priority and dynamic priority adjustment; obtaining task group association relationships through task dependency analysis.

[0062] The scheduling strategy generation module 23 includes generating a multi-dimensional scheduling strategy based on the task requirements. The scheduling strategy includes priority scheduling rules, affinity scheduling rules and resource preemption rules, including: generating priority scheduling rules based on the task priority identifier; generating affinity scheduling rules based on the task group association relationship; generating resource preemption rules based on the resource process status.

[0063] The task allocation module 24 is used to dynamically allocate AI tasks to the target computing nodes of the heterogeneous AI computing resource pool based on the multi-dimensional scheduling strategy, and monitor the task execution status and resource utilization in real time, including: dynamically allocating AI tasks to the target computing nodes based on the multi-dimensional scheduling strategy through a distributed scheduling engine; and collecting task execution status and resource utilization indicators in real time through a unified monitoring interface layer.

[0064] The allocation adjustment module 25 is used to dynamically adjust the computing resource allocation according to the resource utilization, including: generating a dynamic resource utilization adjustment strategy based on the hardware layer, task layer and network layer monitoring function data collected by the unified monitoring interface layer, including resource recovery trigger conditions, fragmented resource merging rules and priority dynamic adjustment model; executing resource adjustment operations through a distributed scheduling engine, including resource recovery and reallocation and task priority rebalancing.

[0065] Figure 3 A schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention, such as Figure 3 As shown, the electronic device 50 includes: a processor 501 (processor), a memory 502 (memory) and a bus 503;

[0066] The processor 501 and the memory 502 communicate with each other via the bus 503 ; the processor 501 is used to call program instructions in the memory 502 to execute the methods provided by the above-mentioned method implementation methods.

[0067] This embodiment provides a non-transitory computer-readable storage medium, which stores computer instructions. The computer instructions enable a computer to execute the methods provided by the above-mentioned method embodiments.

[0068] Those skilled in the art will understand that all or part of the steps for implementing the above-mentioned method implementation method can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method implementation method; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk, etc. Various storage media that can store program codes.

[0069] The device embodiments described above are merely illustrative. Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of these modules may be selected based on actual needs to achieve the objectives of this embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0070] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or certain parts of the embodiment.

[0071] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A heterogeneous AI computing resource scheduling method, characterized in that: The method comprises: Building a heterogeneous AI computing power resource pool that integrates computing resources from multiple heterogeneous AI acceleration chips; Obtaining the scheduling requirements of the AI ​​task, including the task type, resource request amount, priority identifier, and task group association; Generate a multi-dimensional scheduling strategy based on the task requirements, wherein the scheduling strategy includes priority scheduling rules, affinity scheduling rules, and resource preemption rules; Based on the multi-dimensional scheduling strategy, AI tasks are dynamically allocated to target computing nodes in the heterogeneous AI computing resource pool, and task execution status and resource utilization are monitored in real time; The computing resource allocation is dynamically adjusted according to the resource utilization.

2. The heterogeneous AI computing resource scheduling method according to claim 1, characterized in that: The construction of a heterogeneous AI computing resource pool includes: Fine-grained segmentation of the computing cores and memory resources of multiple heterogeneous AI acceleration chips, including at least GPUs, NPUs, and DCUs, is performed. The GPU is segmented using multi-instance GPU-based segmentation technology to divide single-card resources into multiple independent instances; the NPU is segmented using virtualized NPU technology to divide single-card resources into multiple virtual instances; and the DCU is segmented using virtualized DCU technology to divide single-card resources into multiple virtual instances. Abstract and encapsulate the segmented heterogeneous AI acceleration chip resources, shielding the underlying hardware differences and forming standardized resource units; A heterogeneous AI computing power resource pool is constructed based on the standardized resource units.

3. The heterogeneous AI computing resource scheduling method according to claim 1, characterized in that: The scheduling requirements for obtaining AI tasks include: Obtain the task type of the AI ​​task through the task description file or API interface. The task types include: training task, inference task, and edge computing task; Obtain resource requests using resource profiling technology, including video memory requirements, computing core requirements, and communication loan requirements; The priority identifier is generated by user configuration or system automatically, and the priority identifier includes: user-defined priority and dynamic priority adjustment; Obtain task group association relationships through task dependency analysis.

4. The heterogeneous AI computing resource scheduling method according to claim 1, characterized in that: Generating a multi-dimensional scheduling strategy according to the task requirements includes: generating a priority scheduling rule based on the task priority identifier; generating affinity scheduling rules based on the task group association relationship; Generate resource preemption rules based on the resource process status.

5. The heterogeneous AI computing resource scheduling method according to claim 1, characterized in that: The method of dynamically allocating AI tasks to target computing nodes in the heterogeneous AI computing resource pool based on the multi-dimensional scheduling strategy and monitoring task execution status and resource utilization in real time includes: Dynamically assign AI tasks to target computing nodes through a distributed scheduling engine based on the multi-dimensional scheduling strategy; The unified monitoring interface layer collects task execution status and resource utilization indicators in real time.

6. The heterogeneous AI computing resource scheduling method according to claim 1, characterized in that: The dynamically adjusting the computing resource allocation according to the resource utilization comprises: Generate dynamic resource utilization adjustment strategies based on hardware, task, and network layer monitoring data collected by the unified monitoring interface layer, including resource recycling trigger conditions, fragmented resource merging rules, and priority dynamic adjustment models; Resource adjustment operations are performed through the distributed scheduling engine, including resource recovery and reallocation and task priority rebalancing.

7. A heterogeneous AI computing resource scheduling system, characterized by: include: A resource construction module is used to build a heterogeneous AI computing power resource pool that integrates the computing resources of multiple heterogeneous AI acceleration chips; A demand acquisition module is used to obtain the scheduling requirements of AI tasks, which include task type, resource request amount, priority identifier, and task group association relationship; A scheduling strategy generation module, comprising generating a multi-dimensional scheduling strategy according to the task requirements, wherein the scheduling strategy includes priority scheduling rules, affinity scheduling rules and resource preemption rules; The task allocation module is used to dynamically allocate AI tasks to the target computing nodes of the heterogeneous AI computing resource pool based on the multi-dimensional scheduling strategy, and monitor the task execution status and resource utilization in real time.

8. A computer-readable storage medium, characterized in that The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images