Segmentation and scheduling method and system for heterogeneous AI computing power resources

Abstract

The invention discloses a heterogeneous AI computing power resource segmentation and scheduling method and system, and the method comprises the steps: recognizing the type of a heterogeneous AI computing power card accessed to a current server and hardware resource parameters of the heterogeneous AI computing power card, and carrying out the abstract modeling to generate a logic resource model; dividing a computing core and a video memory of a single heterogeneous AI computing power card into at least one independent virtual computing power instance through a resource virtualization technology on the basis of the constraint of a logic resource model; and in response to a computing task request, mounting the virtual computing power instance to a computing task in a containerization form, and dynamically scheduling the virtual computing power instance based on the demand of the computing task. Standardized logic resource units are generated through the minimum segmentation unit, and the utilization rate and scheduling flexibility of heterogeneous AI computing power resources are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of computing resource management, and in particular to a method for dividing and scheduling heterogeneous AI computing resources. Background Art

[0002] Currently, efficient management and scheduling of heterogeneous AI computing resources remains a technical bottleneck in large-scale AI model training and inference scenarios. With the widespread deployment of accelerators with diverse architectures, such as NVIDIA GPUs, Huawei Ascend NPUs, and Hygon DCUs, differences in their hardware characteristics have led to significant divergence in resource partitioning and scheduling methods: NVIDIA uses MIG technology to achieve physical circuit-level isolation, creating up to seven independent instances through exclusive allocation of crossbar ports, L2 cache, and memory controllers; Ascend relies on driver-level virtualization mechanisms, using vir templates to divide AI Cores and graphics memory according to fixed quotas; and Hygon combines hardware minimum granularity with software-based on-demand allocation strategies to achieve uneven partitioning. This fragmented technical approach directly leads to low resource utilization: hardware-level partitioning is limited by preset templates and cannot dynamically adjust idle resources; driver-level virtualization, due to its coarse granularity, is difficult to adapt to small-scale tasks; and hybrid partitioning strategies face the challenge of multi-vendor interface compatibility. What's more serious is that the container scheduler cannot directly perceive the hardware characteristics of heterogeneous AI cards, resulting in frequent mismatches between resource allocation and task requirements. The resource competition and idleness caused by dynamic load changes are difficult to alleviate through elastic scaling mechanisms. Summary of the Invention

[0003] In view of this, the present invention proposes a method and system for partitioning and scheduling heterogeneous AI computing resources, which can cover the complete technical closed loop of "resource modeling-dynamic partitioning-container scheduling". The present invention provides the following technical solutions: A method for partitioning and scheduling heterogeneous AI computing resources, the method comprising: Identify the types of heterogeneous AI computing cards connected to the current server and their hardware resource parameters, and perform abstract modeling to generate a logical resource model; Based on the constraints of the logical resource model, the computing core and video memory of a single heterogeneous AI computing power card are divided into at least one independent virtual computing power instance through resource virtualization technology; In response to a computing task request, the virtual computing power instance is mounted to the computing task in a containerized form, and the virtual computing power instance is dynamically scheduled based on the needs of the computing task.

[0004] Optionally, identifying the type of heterogeneous AI computing power cards connected to the current server and their hardware resource parameters, and performing abstract modeling to generate a logical resource model includes: Automatically identify the type of heterogeneous AI computing power card connected to the current server through the driver interface or hardware identifier. The heterogeneous AI computing power card includes GPUs, NPUs, or DCUs based on different architectures. Collect the physical hardware resource parameters of the heterogeneous AI computing power card, including the number of computing cores, video memory capacity, cache bandwidth, and hardware slicing capability; A logical resource model that is independent of the device type is generated based on the hardware resource parameters, and the logical resource model maps the physical resources of the heterogeneous AI computing power card into a unified logical resource unit.

[0005] Optionally, the dividing of the computing core and video memory of a single heterogeneous AI computing power card into at least one independent virtual computing power instance by resource virtualization technology based on the constraints of the logical resource model includes: Based on the hardware characteristics of heterogeneous AI computing cards, dynamically select any of the following resource virtualization technologies for resource division: Hardware circuit isolation: For AI computing power cards that support hardware-level virtualization, the computing core and video memory are divided into multiple independent instances through circuit path isolation technology, and each instance has its own exclusive physical resource path; Driver-layer virtualization dynamically divides computing cores and video memory for AI computing cards that rely on software scheduling through the driver-layer resource allocation interface; A hybrid sharding strategy combines the hardware minimum sharding unit with the software on-demand allocation strategy for AI computing power cards that support multi-mode virtualization to generate virtual computing power instances of non-uniform granularity. The resource partitioning is constrained according to the minimum segmentation unit preset in the logical resource model.

[0006] Optionally, responding to the computing task request, mounting the virtual computing power instance to the computing task in a containerized form, and dynamically scheduling the virtual computing power instance based on the requirements of the computing task includes: The virtual computing instance is mounted to the target container through the collaboration of the container runtime and the device plug-in, wherein the container runtime includes a dedicated runtime that supports heterogeneous AI computing cards, and the device plug-in includes a resource allocation interface provided by the hardware manufacturer; based on the resource requirements of the computing task, the virtual computing instance that meets the conditions is matched from the logical resource model and allocated to the target container through the scheduling engine; Monitor the load status of virtual computing instances in real time and dynamically adjust resource allocation ratios, including: Elastic scaling: dynamically increase or decrease the number of allocated computing cores and memory capacity based on the peak workload; Priority scheduling, where high-priority tasks preempt the idle resources of low-priority tasks; Affinity scheduling binds compute-intensive tasks to high-bandwidth cache resources, or binds memory-sensitive tasks to low-latency graphics memory resources.

[0007] Optionally, before responding to the computing task request, the method further includes: The virtual computing power instances are classified according to the types of heterogeneous AI computing power cards to construct computing power resource pools, and each computing power resource pool is uniformly managed to form a standardized computing power supply unit.

[0008] Optionally, the method further includes: Physically partitioning the computing core and video memory of the heterogeneous AI computing power card; Each of the virtual computing power instances accesses an allocated exclusive computing core through an independent hardware path; The video memory implements address space isolation through a hardware memory management unit.

[0009] The present invention further discloses a system for dividing and scheduling heterogeneous AI computing resources, including: The model building module is used to identify the type of heterogeneous AI computing power cards connected to the current server and their hardware resource parameters, and perform abstract modeling to generate a logical resource model; An instance partitioning module, configured to partition the computing core and video memory of a single heterogeneous AI computing power card into at least one independent virtual computing power instance based on the constraints of the logical resource model and using resource virtualization technology; The instance mounting module is used to respond to computing task requests, mount the virtual computing power instance to the computing task in a containerized form, and dynamically schedule the virtual computing power instance based on the needs of the computing task.

[0010] The present invention further discloses a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program implements the above method when executed by a processor.

[0011] The present invention further discloses an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above method when executing the program.

[0012] The present invention further discloses a computer program product, comprising a computer program, which implements the above method when executed by a processor.

[0013] According to the technical solution of the present invention, by constructing a logical resource model that is independent of the device type, combined with hardware / software hybrid virtualization technology and a containerized scheduling mechanism, unified abstraction and efficient management of heterogeneous AI computing power cards such as NVIDIA GPUs, Ascend NPUs, and Hygon DCUs are achieved. By identifying the types of heterogeneous computing power cards and collecting their physical resource parameters, standardized logical resource units are generated based on the preset minimum partitioning unit, thereby shielding the underlying hardware differences and forming a unified resource pool; on this basis, hardware circuit isolation, driver layer virtualization, or hybrid partitioning strategies are dynamically selected to achieve resource partitioning and physical path isolation; finally, through the collaboration of the container runtime and the device plug-in, the virtual computing power instance is mounted to the container, and dynamic scheduling strategies such as elastic scaling, priority preemption, and affinity binding are implemented based on task requirements. This significantly improves the utilization rate and scheduling flexibility of heterogeneous AI computing power resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] For purposes of illustration and not limitation, the present invention will now be described with reference to embodiments thereof and the accompanying drawings, in which: Figure 1 This is a flowchart of a method for dividing and scheduling heterogeneous AI computing resources in an embodiment of the present invention; Figure 2 Schematic diagram of the structure of a heterogeneous AI computing resource segmentation and scheduling system in an embodiment of the present invention; Figure 3 Schematic diagram of the structure of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0015] In order to enable those skilled in the art to better understand the solution of this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of this application.

[0016] It should be noted that, in the absence of conflict, the embodiments of the present application and the features thereof can be combined with each other. The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0017] refer to Figure 1 This embodiment discloses a method for partitioning and scheduling heterogeneous AI computing resources. The core of the method is to achieve full lifecycle management of heterogeneous AI acceleration devices such as NVIDIA GPUs, Ascend NPUs, and Hygon DCUs through unified modeling and hybrid virtualization technology. The method includes the following steps: S100: Identify the type of heterogeneous AI computing power cards connected to the current server and their hardware resource parameters, and perform abstract modeling to generate a logical resource model.

[0018] In the parameter identification stage, the type of heterogeneous AI computing power card connected to the current server is first automatically identified through the driver interface or hardware identifier. The heterogeneous AI computing power card includes GPUs, NPUs or DCUs based on different architectures. For example, NVIDIA's NVML interface is called to read the PCIeID and hardware identifier of the A100, the Ascend CANN Toolkit is used to obtain the model parameters of the Ascend 910, or the hdmcli tool of the Hygon DTK is used to query the Z10L device information. On this basis, the physical hardware resource parameters of the heterogeneous AI computing power card are collected, including the number of computing cores, video memory capacity, cache bandwidth and hardware slicing capabilities. Examples include NVIDIA A100's 104 SM cores, 40GB GDDR6 video memory and 2TB / s bandwidth, Ascend 910's 32 AI Cores, 32GB HBM video memory and 1TB / s bandwidth, and Hygon Z10L's 60 CUs, 32GB GDDR6 video memory and 1.5TB / s bandwidth and other key indicators.

[0019] Based on the above-mentioned physical resource parameters, a logical resource model is generated that is independent of the device type. The logical resource model maps the physical resources of heterogeneous AI computing power cards into a unified logical resource unit. For example, NVIDIA's 1SM+5GB video memory constitutes 1 LCU (Logical Compute Unit) and 1 LMU (Logical Memory Unit), Ascend's 1AI Core+8GB video memory corresponds to 1 LCU and 1 LMU, and Hygon's 1CU+1MB video memory forms a finer-grained LCU / LMU combination. During the logical resource model construction process, the system integrates the logical resource units of a single heterogeneous AI computing power card into an independent resource pool and realizes unified management of cross-vendor resource pools. For example, NVIDIA A100's 104LCU+8LMU, Ascend 910's 32LCU+4LMU, and Hygon Z10L's 60LCU+32,768LMU are standardized as schedulable computing power supply units. When a new heterogeneous AI computing card is added, the adapter plug-in automatically loads the vendor driver interface and synchronously updates the logical resource model and resource pool status. This dynamic modeling mechanism not only masks the differences in granularity between NVIDIA MIG's 5GB memory slice, Ascend vNPU's 8GB memory unit, and Hygon vDCU's 1MB memory, but also enables the container scheduler to adapt to heterogeneous resources through abstract design.

[0020] S200: Based on the constraints of the logical resource model, the computing core and video memory of a single heterogeneous AI computing power card are divided into at least one independent virtual computing power instance through resource virtualization technology, and the resource division is constrained according to the minimum division unit preset in the logical resource model. Specifically, based on the hardware characteristics of the heterogeneous AI computing power card, any of the following resource virtualization technologies are dynamically selected for resource division: Hardware circuit isolation, for AI computing power cards that support hardware-level virtualization, uses circuit path isolation technology to divide the computing core and video memory into multiple independent instances, each instance has its own physical resource path. The exemplary NVIDIA computing power card parses the MIG instance specification template through the nvidia-smi command, and physically partitions the computing core (SM) and video memory in a fixed ratio at the hardware driver layer. For example, when an NVIDIA A100 GPU (80 SM, 40 GB video memory) is identified, 7 1g.5gb instances are automatically created based on the maximum instance number constraint of 7 in the logical resource model. Each instance has 14 SMs and 5 GB video memory, and hardware path isolation is achieved through independent ports and L2 cache.

[0021] Driver-layer virtualization dynamically divides computing cores and video memory for AI computing power cards that rely on software scheduling through the driver-layer resource allocation interface. For example, the Ascend NPU and Hygon DCU, where the Ascend NPU matches a predefined virtualization template (such as vir16), uses the CANN toolkit to divide the number of AICore cores (16 cores) and memory (16GB) of the physical NPU at the driver layer, generating a vNPU instance and mounting it in the container. The Hygon DCU allocates computing cores and video memory on demand through the hdmcli tool. For example, the Z100LDCU (60CU, 32GB video memory) is divided into 4 vDCU instances, and the video memory address space is forcibly isolated by the hardware MMU.

[0022] A hybrid sharding strategy combines hardware minimum sharding units with software on-demand allocation for AI computing cards that support multi-mode virtualization, generating non-uniformly sized virtual computing instances. When the logical resource model detects that the chip supports multi-mode virtualization, it prioritizes building a base instance using the hardware minimum sharding unit. The driver interface then uses the memory resource to split the instance into smaller units based on task requirements. All virtual computing instances are equipped with fault isolation mechanisms, such as dynamic driver destruction of NVIDIA MIG instances in the event of anomalies; resource release through Ascend vNPUs and container circuit breaking; and automatic recovery of Hygon vDCU processes by HDMS tools in the event of process crashes.

[0023] Therefore, through the unified scheduling of three technical paths—hardware circuit isolation, driver layer templates, and on-demand allocation—we can effectively improve single-GPU utilization while ensuring resource exclusivity. Furthermore, we strictly adhere to the maximum number of shardable instances and minimum resource granularity in the logical resource model to avoid over-physical resource allocation. Furthermore, virtual instances ensure exclusive access to the compute cores through independent hardware paths, and memory is isolated through hardware MMU addresses to prevent out-of-bounds operations, fundamentally addressing multitasking interference.

[0024] S300: Classify the virtual computing power instances according to the types of heterogeneous AI computing power cards to construct computing power resource pools, and uniformly manage each computing power resource pool to form a standardized computing power supply unit.

[0025] In this embodiment, after completing the division of virtual computing power instances, the system performs resource pooling operations based on the hardware type identification of the heterogeneous AI computing power cards. By classifying the heterogeneous AI computing power cards obtained above into physical resource pools of corresponding types, for example: GPU resource pool: stores all MIG sharding instances; NPU resource pool: collects Ascend vNPU instances; DCU resource pool: aggregates Hygon vDCU instances.

[0026] Each resource pool independently maintains its instance specification catalog and prohibits cross-pool scheduling through hardware affinity policies. Furthermore, a multi-pool coordination controller is deployed on the management platform, enabling unified management and control through driver interfaces: NVIDIA pools call the NVML API to obtain instance status, Ascend pools connect to the CANN Toolkit to synchronize vNPU resources, and Hygon pools connect to the hdmcli tool to monitor the vDCU lifecycle. Furthermore, standardized computing power provision units are dynamically generated to map heterogeneous instance parameters to a unified unit of measure. A computing power provision unit is a standard compute core.

[0027] S400: In response to a computing task request, the virtual computing power instance is mounted to the computing task in a containerized form, and the virtual computing power instance is dynamically scheduled based on the requirements of the computing task.

[0028] When a computing task request arrives, the scheduling engine performs dynamic scheduling based on the task type and resource demand characteristics. First, the virtual computing power instance is mounted to the target container through the hardware manufacturer's dedicated container runtime, allowing the application in the container to directly access the underlying hardware instruction set. In this embodiment, the task types include training tasks and inference tasks, where the training tasks are scheduled using hardware affinity, by automatically configuring the cross-instance communication topology. If the task requires cross-card communication, multiple virtual instances are bound to the same physical card or adjacent cards interconnected by NVLink / HCCS to generate a hardware-level collective communication group. For inference tasks, an over-resolution scheduling strategy is adopted to concurrently schedule multiple low-load tasks to a single instance based on task priority. In addition, real-time resource regulation needs to be performed. Specifically, when the memory peak value exceeds the threshold, additional memory is automatically allocated from the resource pool; if a MIG instance ECC memory error is detected, the faulty instance is immediately destroyed and a new instance is mounted to the task container; high-priority inference tasks can preempt the idle computing cores of low-priority tasks.

[0029] S500: Physically partition the computing core and video memory of the heterogeneous AI computing power card; each virtual computing power instance accesses the allocated exclusive computing core through an independent hardware path; and implements address space isolation of the video memory through a hardware memory management unit.

[0030] During the creation of a virtual computing instance, the system uses the native hardware capabilities of the chip to achieve physical isolation between the computing core and the video memory. Specifically, the following example shows the physical partitioning and exclusive path of the computing core: NVIDIA GPU: Based on MIG technology, the computing cores of the physical GPU are divided into fixed specifications. Each instance accesses an exclusive SM through an independent crossbar port to ensure hardware isolation of the instruction execution path.

[0031] Ascend NPU: The number of AI Cores is physically partitioned within the chip based on the virtualization template. Each vNPU instance accesses 16 AI Cores through a dedicated computing unit allocated by the Ascend chip scheduler, preventing cross-instance computing interference.

[0032] Hygon DCU: The computing core is divided on demand through the hdmcli tool. Each vDCU instance is bound to an independent hardware queue to achieve physical channel isolation of computing instructions.

[0033] Further hard isolation of the video memory address space: NVIDIA GPU: The DRAM address bus is divided by instance, and hard isolation is used by the memory controller to prevent out-of-bounds access.

[0034] Hygon DCU: Video memory is allocated at MB granularity, and the hardware MMU is used to forcibly isolate the address space. Single-instance video memory overflow only triggers local exceptions.

[0035] Ascend NPU: vNPU memory isolation is achieved through memory controller partitioning, and illegal address access is blocked in conjunction with the CANN toolkit.

[0036] Build a fault isolation linkage mechanism: Hardware error signals are confined to the instance partition to which they belong, triggering automatic recovery at the driver layer: NVIDIA MIG instances are dynamically destroyed and rebuilt through nvidia-smi; Hygon vDCU resources are reclaimed by the HDMS tool; Ascend vNPU executes container circuit breaking and releases AICore resources.

[0037] refer to Figure 2 This embodiment further discloses a system for dividing and scheduling heterogeneous AI computing resources, which includes: The model building module 21 is used to identify the type of heterogeneous AI computing power card connected to the current server and its hardware resource parameters, and perform abstract modeling to generate a logical resource model; including: automatically identifying the type of heterogeneous AI computing power card connected to the current server through a driver interface or hardware identifier, the heterogeneous AI computing power card includes a GPU, NPU or DCU based on different architectures; collecting the physical hardware resource parameters of the heterogeneous AI computing power card, including the number of computing cores, video memory capacity, cache bandwidth and hardware splitting capability; generating a logical resource model that is independent of the device type based on the hardware resource parameters, and the logical resource model maps the physical resources of the heterogeneous AI computing power card into a unified logical resource unit.

[0038] The instance partitioning module 22 is used to divide the computing core and video memory of a single heterogeneous AI computing power card into at least one independent virtual computing power instance based on the constraints of the logical resource model through resource virtualization technology; including: dynamically selecting any of the following resource virtualization technologies for resource partitioning according to the hardware characteristics of the heterogeneous AI computing power card: hardware circuit isolation, for AI computing power cards that support hardware-level virtualization, the computing core and video memory are divided into multiple independent instances through circuit path isolation technology, and each instance has exclusive physical resource paths; driver layer virtualization, for AI computing power cards that rely on software scheduling, the computing core and video memory are dynamically divided through the driver layer resource allocation interface; hybrid splitting strategy, for AI computing power cards that support multi-mode virtualization, the hardware minimum splitting unit and the software on-demand allocation strategy are combined to generate virtual computing power instances of non-uniform granularity; the resource partitioning is constrained according to the minimum splitting unit preset in the logical resource model.

[0039] The computing power resource pool construction module 23 is used to classify the virtual computing power instances according to the types of heterogeneous AI computing power cards to construct computing power resource pools, and to uniformly manage each computing power resource pool to form a standardized computing power supply unit.

[0040] The instance mounting module 24 is used to respond to computing task requests, mount the virtual computing power instance to the computing task in a containerized form, and dynamically schedule the virtual computing power instance based on the needs of the computing task; including: mounting the virtual computing power instance to the target container through the container runtime and the device plug-in, wherein the container runtime includes a dedicated runtime that supports heterogeneous AI computing power cards, and the device plug-in includes a resource allocation interface provided by the hardware manufacturer; based on the resource requirements of the computing task, matching the virtual computing power instance that meets the conditions from the logical resource model, and allocating it to the target container through the scheduling engine; real-time monitoring of the load status of the virtual computing power instance, and dynamically adjusting the resource allocation ratio, including: elastic scaling, dynamically increasing or decreasing the number of allocated computing cores and video memory capacity according to the task load peak; priority scheduling, preempting the idle resources of low-priority tasks for high-priority tasks; affinity scheduling, binding computing-intensive tasks with high-bandwidth cache resources, or binding memory-sensitive tasks with low-latency video memory resources.

[0041] The spatial isolation module 25 is used to physically partition the computing core and video memory of the heterogeneous AI computing power card; each virtual computing power instance accesses the allocated exclusive computing core through an independent hardware path; and the address space isolation of the video memory is implemented through the hardware memory management unit.

[0042] Figure 3 A schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention, such as Figure 3 As shown, the electronic device 50 includes: a processor 501 (processor), a memory 502 (memory) and a bus 503; The processor 501 and the memory 502 communicate with each other via the bus 503 ; the processor 501 is used to call program instructions in the memory 502 to execute the methods provided by the above-mentioned method implementation methods.

[0043] This embodiment provides a non-transitory computer-readable storage medium, which stores computer instructions. The computer instructions enable a computer to execute the methods provided by the above-mentioned method embodiments.

[0044] Those skilled in the art will understand that all or part of the steps for implementing the above-mentioned method implementation method can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method implementation method; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk, etc. Various storage media that can store program codes.

[0045] The device embodiments described above are merely illustrative. Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of these modules may be selected based on actual needs to achieve the objectives of this embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0046] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or certain parts of the embodiment.

[0047] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A method for dividing and scheduling heterogeneous AI computing resources, characterized in that: The method comprises: Identify the types of heterogeneous AI computing cards connected to the current server and their hardware resource parameters, and perform abstract modeling to generate a logical resource model; Based on the constraints of the logical resource model, the computing core and video memory of a single heterogeneous AI computing power card are divided into at least one independent virtual computing power instance through resource virtualization technology; In response to a computing task request, the virtual computing power instance is mounted to the computing task in a containerized form, and the virtual computing power instance is dynamically scheduled based on the needs of the computing task.

2. The method for dividing and scheduling heterogeneous AI computing resources according to claim 1 is characterized in that: Identifying the type of heterogeneous AI computing power cards connected to the current server and their hardware resource parameters, and performing abstract modeling to generate a logical resource model includes: automatically identifying the type of heterogeneous AI computing power cards connected to the current server through a driver interface or hardware identifier, wherein the heterogeneous AI computing power cards include GPUs, NPUs, or DCUs based on different architectures; Collect the physical hardware resource parameters of the heterogeneous AI computing power card, including the number of computing cores, video memory capacity, cache bandwidth, and hardware slicing capability; A logical resource model that is independent of the device type is generated based on the hardware resource parameters, and the logical resource model maps the physical resources of the heterogeneous AI computing power card into a unified logical resource unit.

3. The method for dividing and scheduling heterogeneous AI computing resources according to claim 1 is characterized in that: The method of dividing the computing core and video memory of a single heterogeneous AI computing power card into at least one independent virtual computing power instance based on the constraints of the logical resource model through resource virtualization technology includes: Based on the hardware characteristics of heterogeneous AI computing cards, dynamically select any of the following resource virtualization technologies for resource division: Hardware circuit isolation: For AI computing power cards that support hardware-level virtualization, the computing core and video memory are divided into multiple independent instances through circuit path isolation technology, and each instance has its own exclusive physical resource path; Driver-layer virtualization dynamically divides computing cores and video memory for AI computing cards that rely on software scheduling through the driver-layer resource allocation interface; A hybrid sharding strategy combines the hardware minimum sharding unit with the software on-demand allocation strategy for AI computing power cards that support multi-mode virtualization to generate virtual computing power instances of non-uniform granularity. The resource partitioning is constrained according to the minimum segmentation unit preset in the logical resource model.

4. The method for dividing and scheduling heterogeneous AI computing resources according to claim 1 is characterized in that: The step of responding to the computing task request, mounting the virtual computing instance to the computing task in a containerized form, and dynamically scheduling the virtual computing instance based on the requirements of the computing task includes: The virtual computing instance is mounted to the target container through the collaboration of the container runtime and the device plug-in, wherein the container runtime includes a dedicated runtime that supports heterogeneous AI computing cards, and the device plug-in includes a resource allocation interface provided by the hardware manufacturer; based on the resource requirements of the computing task, the virtual computing instance that meets the conditions is matched from the logical resource model and allocated to the target container through the scheduling engine; Monitor the load status of virtual computing instances in real time and dynamically adjust resource allocation ratios, including: Elastic scaling: dynamically increase or decrease the number of allocated computing cores and memory capacity based on the peak workload; Priority scheduling, where high-priority tasks preempt the idle resources of low-priority tasks; Affinity scheduling binds compute-intensive tasks to high-bandwidth cache resources, or binds memory-sensitive tasks to low-latency graphics memory resources.

5. The method for dividing and scheduling heterogeneous AI computing resources according to claim 1 is characterized in that: Before responding to the computing task request, the method further includes: The virtual computing power instances are classified according to the types of heterogeneous AI computing power cards to construct computing power resource pools, and each computing power resource pool is uniformly managed to form a standardized computing power supply unit.

6. The method for dividing and scheduling heterogeneous AI computing resources according to any one of claim 1, characterized in that: The method further comprises: Physically partitioning the computing core and video memory of the heterogeneous AI computing power card; Each of the virtual computing power instances accesses an allocated exclusive computing core through an independent hardware path; The video memory implements address space isolation through a hardware memory management unit.

7. A system for dividing and scheduling heterogeneous AI computing resources, characterized by: include: The model building module is used to identify the type of heterogeneous AI computing power cards connected to the current server and their hardware resource parameters, and perform abstract modeling to generate a logical resource model; An instance partitioning module, configured to partition the computing core and video memory of a single heterogeneous AI computing power card into at least one independent virtual computing power instance based on the constraints of the logical resource model and using resource virtualization technology; The instance mounting module is used to respond to computing task requests, mount the virtual computing power instance to the computing task in a containerized form, and dynamically schedule the virtual computing power instance based on the needs of the computing task.

8. A computer-readable storage medium, characterized in that The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images