AI computing resource management method and system thereof

Abstract

The application discloses an AI computing resource management method and system, relates to the technical field of computing resource management, and comprises the following steps: step S1, priority is set and the priority is adjusted; step S2, a real-time dynamic adjustment strategy is realized through an LLM scheduler; step S3, a basic quota and a shared quota are allocated, and the shared quota is allocated based on segmentation; step S4, devices with continuous numbers are forced to be allocated, high-priority tasks preempt low-priority resources, and states are automatically saved; and step S5, members and structures are automatically updated, and pre-processing is used instead of real-time topology calculation. According to the application, high-priority basic quotas and shared quotas are more, low-priority basic quotas and shared quotas are less, and interval distribution is realized, so that the resource intervals of different priorities are both layered and staggered, interval distribution is realized, fragmentation of quota resources is avoided, shared basic quota intervals are realized, resource utilization is improved, and balance between guarantee and efficiency of resource allocation is realized.

Description

Technical Field

[0001] This invention relates to the field of computing resource management technology, and in particular to an AI computing resource management method and system. Background Technology

[0002] AI computing resource management is a method and system specifically designed for scheduling, allocating, monitoring, and optimizing AI-specific computing resources such as GPUs, TPUs, and NPUs. Based on task requirements and priorities, it allocates appropriate computing power to tasks, resolving the contradiction between scarce, expensive, and highly exclusive AI chips and the heterogeneous, variable, and large-scale AI application needs, and ensuring that computing power can play its maximum value.

[0003] Traditional computing resource management methods follow a passive, long chain of "perception-analysis-decision-execution." Schedulers rely on preset rules or static priorities. When faced with heterogeneous hardware failures, complex resource fragmentation, or multi-objective optimization involving overlapping energy consumption and costs, the performance of dynamically adjusting strategies based on real-time load changes is poor. This often results in scheduling decisions failing to reach the optimal solution, thus lengthening the overall waiting time of tasks and reducing efficiency and real-time performance. Furthermore, when processing multiple tasks, progress is easily lost when tasks are preempted or interrupted due to failures. After tasks are resumed, computation must start from scratch, resulting in a significant waste of computing power on repetitive tasks and further reducing efficiency. Summary of the Invention

[0004] The purpose of this invention is to provide an AI computing power resource management method and system to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: an AI computing power resource management method, comprising the following steps: Step S1: Based on different environments and departments, deploy them in separate workspaces, set priorities, and adjust the priorities in real time according to historical data; Step S2: Receive and understand the natural language intent of the instructions through the LLM scheduler, dynamically adjust the strategy based on real-time feedback, verify the adjusted strategy, and filter the decision trajectory. Step S3: Allocate weight according to priority, allocate a basic quota to ensure basic operations, and allocate quotas exceeding the basic quota as shared quotas. The shared quotas are set between adjacent priority segments based on a segmented allocation method, serving as resources for allocation between priorities. Step S4: The scheduler identifies the node type and, according to the quota allocation method in step S3, calls the basic quota and shared quota of the corresponding priority, forcibly requiring the allocation of consecutively numbered devices to ensure that tasks are in the same high-speed interconnect domain. At the same time, when the quota of a high-priority computing task cannot meet the requirements, the low-priority computing task is temporarily interrupted, resources are released, the interrupted task is automatically saved and queued, waiting to resume execution, and when a low-priority task starts, data is automatically loaded from the latest checkpoint. Step S5: Construct a multi-layered topology based on the labels on the nodes. The controller automatically updates the members and structure to ensure that the topology information is consistent with the cluster state. At the same time, during the scheduling process, preprocessing replaces real-time topology calculation, allowing the scheduler to read the results directly from the cache.

[0006] Preferably, the priority setting in step S1 includes the following steps: Step S11: Establish a unified priority hierarchy standard based on business scenarios, and create independent queues for different environments and departments. Each queue defines the resource quota that the workspace can obtain. Step S12: At the queue level, a preemption mechanism is used to ensure that high-priority tasks can obtain resources, and high-priority tasks can preempt resources from low-priority tasks.

[0007] Preferably, the priority adjustment in step S1 includes the following steps: Step S13: Track the historical resource usage of each queue through the scheduler and dynamically adjust the priority. When a queue uses more resources, its priority will be dynamically reduced when allocating shared resources in the future. Step S14: The longer the pause waiting time is, the higher the priority of the queue, and its priority rises to a level sufficient to obtain resources.

[0008] Preferably, the dynamic adjustment strategy in S2 includes the following steps: Step S21: The LLM scheduler automatically generates a queue list and arranges it in order by translating human-readable instructions into machine-executable scheduling policies; Step S22: Based on the queue list order in Step S21, when an error occurs in the previous queue, the LLM scheduler skips that queue and continues to try in the next queue using a binary backoff strategy: Step S23: Store failure mode and success strategy data in a vector database to provide reference data for subsequent scheduling.

[0009] Preferably, the resource quota allocation in step S3 includes the following steps: Step S31: Allocate resources proportionally according to priority, with higher priority tasks receiving more resources than lower priority tasks. Allocate basic quotas to maintain different priority tasks, and use quotas exceeding the sum of basic quotas for different priorities as shared quotas for different priority tasks. Step S32: The basic quotas of different priorities in step S31 are distributed in a segmented manner, and the shared quotas are distributed among different priorities according to the priority ratio. The basic quotas of different priorities and the allocated shared quotas are distributed at intervals. Step S33: When a high-priority task is being processed, the basic quota corresponding to the priority is called first. If the basic quota is insufficient for the operation, the shared quota between the high-priority task and the next lower priority task is called.

[0010] Preferably, the allocation of consecutive numbering devices in step S4 includes the following steps: Step S41: The scheduler obtains the hardware characteristics of the nodes through the node label or feature discovery mechanism. After identifying the node type, the scheduler calls the preset topology policy and uses the basic quota and shared quota of the corresponding priority. Step S42: Utilize the collaborative work of the scheduler and the mesh topology to continuously index in the physical loop, determine the physical location number of the GPU, determine the allocation combination under different GPU quantity requests based on hardware interconnect rules, index to complete contiguous blocks that meet the conditions, and the scheduler writes this allocation result into the annotation of the scheduling unit.

[0011] Preferably, the priority calculation in step S4 includes the following steps: Step S43: After a high-priority task is submitted, the scheduler detects high-priority computing resources. When the detected resources are insufficient, it simulates preemption and selects an eviction target, triggers a low-priority task to save a snapshot and evicts the low-priority task, and schedules the high-priority task. Step S44: Combining the resource reservation strategy, after a task is preempted, some metadata is retained. When the high-priority task in step S43 is completed, the preempted resources are released and the task is restored first. When the task starts, it automatically continues to load data from the last checkpoint before the interruption.

[0012] Preferably, the multi-layer topology update in step S5 includes the following steps: Step S51: By defining the correspondence between topology levels and node labels in the controller, the controller will periodically scan the cluster nodes and automatically construct a multi-layered HyperNode topology based on the labels on the nodes. Step S52: When a node's label changes, or a node is added or removed, the controller automatically updates the HyperNode's members and structure to ensure that the topology information is always consistent with the cluster state and that the topology information is updated in real time.

[0013] Preferably, the preprocessing in step S5 includes the following steps: Step S53: By pre-calculating the topology score of the GPU on the node and storing the assigned quality score and topology loss score on the node, the scheduler directly reads the pre-calculated score when making a decision, avoiding the overhead of real-time topology calculation; Step S54: The scheduler reads and caches the static topology information once at startup, and subsequent scheduling decisions are read directly from the cache.

[0014] An AI computing power resource management system, specifically including: The hardware adaptation module is used to ensure compatibility with AI chips, shield the differences in hardware drivers and instruction sets, and transform physical resources into a unified schedulable logical resource pool. The resource allocation module uses queues as the basic unit of resource scheduling, sets different resource quotas in multi-level queues, achieves physical isolation for different environments, departments, and sets queue priorities; The quota scheduling module performs fine-grained scheduling of resource allocation based on topology-aware scheduling and dynamic priority reduction. The AI ​​management module is used to logically isolate and control access to personnel, resources and data, and provides a unified database; A data storage module is provided for caching topology calculation data and task data. The sensory interaction module is used to comprehensively monitor hardware, communication, and AI task parameters. The sensory interaction module, hardware adaptation module, resource allocation module, quota scheduling module, AI management module, and data storage module are electrically connected.

[0015] The technical effects and advantages of this invention are as follows: This invention utilizes a topology-aware scheduling method to allocate resources in a segmented manner, forming a gradient-interval distribution with higher priority basic quotas and more shared quotas, and lower priority basic quotas and fewer shared quotas. Simultaneously, it ensures that resource intervals of different priorities are both hierarchical and staggered, with interval distribution to avoid quota fragmentation. Shared basic quota intervals improve resource utilization, forming an integrated design of hierarchical protection, proportional sharing, and staggered intervals. Basic quotas are segmented and isolated according to priority to form a safety net, while shared quotas are allocated proportionally according to priority to form a growth line. The superposition of these two methods ensures that resource intervals of different priorities remain continuous yet overlap, achieving a balance between resource allocation protection and efficiency. Furthermore, after a task is preempted, some metadata is retained. After a high-priority task completes its computation, the preempted task is prioritized for resumption after the resources are released. When a task starts, it automatically resumes loading data from the last checkpoint before interruption, avoiding recalculation from the beginning, preventing wasted computing power, and improving computational efficiency. Attached Figure Description

[0016] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention, but do not constitute a limitation thereof. In the drawings: Figure 1 This is a schematic diagram of the AI ​​computing resource management method of the present invention; Figure 2 This is a schematic diagram of the priority setting process of the present invention; Figure 3 This is a schematic diagram of the priority adjustment process of the present invention; Figure 4 This is a schematic diagram of the dynamic adjustment strategy process of the present invention; Figure 5 This is a schematic diagram of the resource quota allocation process of the present invention; Figure 6 This is a schematic diagram of the process for allocating consecutive numbering devices according to the present invention; Figure 7 This is a schematic diagram of the priority calculation process of the present invention; Figure 8 This is a schematic diagram of the multi-layer topology update process of the present invention; Figure 9 This is a schematic diagram of the pre-processing flow of the present invention; Figure 10 This is a schematic diagram of the AI ​​computing power resource management system framework of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] This invention provides, for example Figures 1-9 The AI ​​computing resource management method shown includes the following steps: Step S1: Based on different environments and departments, deploy them in separate workspaces, set priorities, and adjust the priorities in real time according to historical data; Step S2: Receive and understand the natural language intent of the instructions through the LLM scheduler, dynamically adjust the strategy based on real-time feedback, verify the adjusted strategy, and filter the decision trajectory. Step S3: Allocate weight according to priority, allocate a basic quota to ensure basic operations, and allocate quotas exceeding the basic quota as shared quotas. The shared quotas are set between adjacent priority segments based on a segmented allocation method, serving as resources for allocation between priorities. Step S4: The scheduler identifies the node type and, according to the quota allocation method in step S3, calls the basic quota and shared quota of the corresponding priority, forcibly requiring the allocation of consecutively numbered devices to ensure that tasks are in the same high-speed interconnect domain. At the same time, when the quota of a high-priority computing task cannot meet the requirements, the low-priority computing task is temporarily interrupted, resources are released, the interrupted task is automatically saved and queued, waiting to resume execution, and when a low-priority task starts, data is automatically loaded from the latest checkpoint. Step S5: Construct a multi-layered topology based on the labels on the nodes. The controller automatically updates the members and structure to ensure that the topology information is consistent with the cluster state. At the same time, during the scheduling process, preprocessing replaces real-time topology calculation, allowing the scheduler to read the results directly from the cache.

[0019] A topology-aware scheduling method is used to allocate resources in segments, distributing them proportionally based on priority. This allocation includes guaranteed quotas and shared quotas, with shared quotas set between adjacent priority segments. The system first calls the basic quota for the corresponding priority. If the basic quota is insufficient for computation, the shared quota between the higher and next-lower priority segments is called. If this still fails, the front-end scheduling unit of the next-lower priority basic quota is preempted, ensuring tasks remain within the same high-speed interconnect domain. This allows for fine-tuning of resource allocation. Furthermore, after a task is preempted, some metadata is retained. Once the higher-priority task completes its computation, the preempted task is restored first after the resources are released. When a task starts, it automatically resumes loading data from the last checkpoint before the interruption, avoiding recalculation from the beginning, preventing wasted computing power, and improving computational efficiency.

[0020] Setting the priority in step S1 includes the following steps: Step S11: Establish a unified priority hierarchy standard based on business scenarios, and create independent queues for different environments and departments. Each queue defines the resource quota that the workspace can obtain. Step S12: At the queue level, a preemption mechanism is used to ensure that high-priority tasks can obtain resources, and high-priority tasks can preempt resources from low-priority tasks.

[0021] Queues can be defined by environment as production queues, testing queues, and development queues; and by department as algorithm department queues, business department queues, and operations and maintenance department queues. Each department or environment should have an independent queue, and resource boundaries should be defined at the queue level. The scheduler provides a three-level quota mechanism: minimum resource guarantee, normal target value, and hard upper limit. The minimum resource guarantee is the bottom line for core business, the normal target value is the benchmark for fair allocation, and the hard upper limit is the range that cannot be exceeded. Each queue should be bound to a priority range, which limits the highest priority that tasks in the queue can use. The preemption strategy is used to ensure that high-priority tasks obtain resources when resources are scarce. Under the same queue quota, high-priority tasks are allowed to preempt resources from low-priority tasks.

[0022] The priority adjustment in step S1 includes the following steps: Step S13: Track the historical resource usage of each queue through the scheduler and dynamically adjust the priority. When a queue uses more resources, its priority will be dynamically reduced when allocating shared resources in the future. Step S14: The longer the pause waiting time is, the higher the priority of the queue, and its priority rises to a level sufficient to obtain resources.

[0023] The scheduler records historical data on resource consumption for each queue. New tasks in queues that consume more resources have lower priority. In other words, the more resources a queue calls, the lower its priority. This prevents a queue from occupying quota resources for a long time. Furthermore, if a task waits in a queue for too long, the scheduler will continuously increase its priority until it finally gets the resources.

[0024] The dynamic adjustment strategy in S2 includes the following steps: Step S21: The LLM scheduler automatically generates a queue list and arranges it in order by translating human-readable instructions into machine-executable scheduling policies; Step S22: Based on the queue list order in Step S21, when an error occurs in the previous queue, the LLM scheduler skips that queue and continues to try in the next queue using a binary backoff strategy: Step S23: Store failure mode and success strategy data in a vector database to provide reference data for subsequent scheduling.

[0025] The LLM scheduler breaks down input command text into structured intents, then transforms these intents into machine-executable scheduling policies. It automatically generates priority queues for execution. When resource allocation fails, the LLM scheduler intelligently adjusts its policies, employing a more refined approach to explore available resources. LLM first analyzes and understands the semantics of the errors, then uses a binary backoff strategy to continue attempting to call possible resources within the queue. If the number of consecutive errors exceeds a preset value, the queue is skipped, and the next queue is explored until all needs are met. Simultaneously, leveraging the LLM scheduler's long-term memory and experience-based learning capabilities, a complete learning loop is formed, encompassing experience vectorization, similarity retrieval, and policy reuse. This transforms the success or failure of each scheduling attempt into searchable and referable knowledge, making subsequent scheduling decisions increasingly intelligent and faster, thus improving the efficiency and real-time performance of resource allocation.

[0026] The LLM scheduler uses a large language model as the core of the scheduling system. It understands the user's natural language instructions, dynamically generates actions based on the real-time context natural language description, analyzes the current environmental state, infers and formulates a scheduling plan, and finally executes the action, learning and improving from the results.

[0027] The resource quota allocation in step S3 includes the following steps: Step S31: Allocate resources proportionally according to priority, with higher priority tasks receiving more resources than lower priority tasks. Allocate basic quotas to maintain different priority tasks, and use quotas exceeding the sum of basic quotas for different priorities as shared quotas for different priority tasks. Step S32: The basic quotas of different priorities in step S31 are distributed in a segmented manner, and the shared quotas are distributed among different priorities according to the priority ratio. The basic quotas of different priorities and the allocated shared quotas are distributed at intervals. Step S33: When a high-priority task is being processed, the basic quota corresponding to the priority is called first. If the basic quota is insufficient for the operation, the shared quota between the high-priority task and the next lower priority task is called.

[0028] Based on priority stratification, a basic quota is first guaranteed to ensure that each priority task has a minimum resource guarantee. The remaining resources are then allocated as a shared pool on demand. High-priority tasks enjoy a larger share in both stratified allocations. High-priority tasks not only have an advantage in basic guarantees but also have a larger share when competing for additional resources. The basic quota is guaranteed in segments according to priority, and the shared quota is allocated proportionally. This results in a gradient distribution with high-priority tasks having more basic and shared quotas and low-priority tasks having fewer basic and shared quotas. At the same time, it ensures that the resource intervals of different priorities are both stratified and staggered. When allocating computing resources, each priority priority uses the basic quota first, and low-priority tasks are not infinitely squeezed out by high-priority tasks. High-priority tasks have an advantage when competing for additional resources. The interval distribution avoids the fragmentation of quota resources. The shared basic quota interval improves resource utilization. This forms an integrated design of stratified guarantee, proportional sharing, and interval staggering. The basic quota is segmented and isolated according to priority to form a safety net, and the shared quota is allocated proportionally according to priority to form a growth line. After the two are combined, the resource intervals of each priority remain continuous and overlapping, achieving a balance between resource allocation guarantee and efficiency.

[0029] The allocation of consecutive numbering devices in step S4 includes the following steps: Step S41: The scheduler obtains the hardware characteristics of the nodes through the node label or feature discovery mechanism. After identifying the node type, the scheduler calls the preset topology policy and uses the basic quota and shared quota of the corresponding priority. Step S42: Utilize the collaborative work of the scheduler and the mesh topology to continuously index in the physical loop, determine the physical location number of the GPU, determine the allocation combination under different GPU quantity requests based on hardware interconnect rules, index to complete contiguous blocks that meet the conditions, and the scheduler writes this allocation result into the annotation of the scheduling unit.

[0030] The detected information is converted into node labels through NFD. These standardized labels are the primary basis for the scheduler to identify node types. After the scheduler identifies a node by its label, it calls the preset strategy for such high-performance nodes. The main goal is to ensure the communication efficiency of multi-GPU training tasks. When a task requires a corresponding number of GPUs, the scheduler will not arbitrarily allocate combinations, but will force the allocation of complete contiguous blocks that meet the task requirements, ensuring that all GPUs are in the same NVLink domain and avoiding performance loss caused by cross-domain communication.

[0031] NFD is an open-source project from the official Kubernetes community. It is used to automatically detect the hardware and system configuration characteristics of each node in the cluster and use this information as a node label. An NVLink domain refers to a set of GPUs that are directly connected through NVLink high-speed interconnect technology. GPUs in the same domain can enjoy extremely high communication bandwidth and extremely low latency.

[0032] The priority calculation in step S4 includes the following steps: Step S43: After a high-priority task is submitted, the scheduler detects high-priority computing resources. When the detected resources are insufficient, it simulates preemption and selects an eviction target, triggers a low-priority task to save a snapshot and evicts the low-priority task, and schedules the high-priority task. Step S44: Combining the resource reservation strategy, after a task is preempted, some metadata is retained. When the high-priority task in step S43 is completed, the preempted resources are released and the task is restored first. When the task starts, it automatically continues to load data from the last checkpoint before the interruption.

[0033] When a high-priority task is in progress, the scheduler uses simulation to find a low-priority task that can preempt resources, saves the progress of the low-priority task, and then frees up the resources of the low-priority task for the high-priority task to ensure its execution. Simultaneously, when preempting resources from a low-priority task, the low-priority task is paused and its training state is saved, along with the metadata needed for recovery. When the high-priority task completes, the system returns the resources to the shared quota and prioritizes reserving them for the preempted low-priority task. The scheduler checks the preemption recovery queue, finds that the preempted low-priority task is waiting for recovery and meets its resource requirements, marks the resource as reserved, prevents it from entering the shared quota, and immediately initiates the low-priority task recovery process, automatically loading the previously saved metadata to avoid recalculating the task from scratch, preventing wasted computing power and improving computational efficiency.

[0034] The multi-layer topology update in step S5 includes the following steps: Step S51: By defining the correspondence between topology levels and node labels in the controller, the controller will periodically scan the cluster nodes and automatically construct a multi-layered HyperNode topology based on the labels on the nodes. Step S52: When a node's label changes, or a node is added or removed, the controller automatically updates the HyperNode's members and structure to ensure that the topology information is always consistent with the cluster state and that the topology information is updated in real time.

[0035] The scheduling system automatically discovers, builds, and updates the hierarchical topology of the cluster through the controller, ensuring that the scheduler always makes decisions based on the latest and most accurate hardware layout. The physical or logical hierarchical structure of the data center and cluster is defined at the topology level. HyperNode abstracts the above physical topology into a logical structure that the scheduler can understand. In the controller, the configuration file defines how labels are mapped to the topology level. NFD assigns corresponding labels to nodes. The controller periodically scans the cluster nodes, automatically builds a multi-level HyperNode topology tree according to the preset label rules, and responds to the addition, deletion, and changes of nodes in real time.

[0036] The controller refers to the topology management controller, which is responsible for monitoring the cluster status and ensuring that the actual data of the cluster always conforms to the expected topology. HyperNode is a custom resource used to solve the problem of how to accurately schedule computing tasks to the most efficient devices in scenarios such as AI large model training. It is a standardized and hierarchical abstract description of the data center network topology.

[0037] The preprocessing in step S5 includes the following steps: Step S53: By pre-calculating the topology score of the GPU on the node and storing the assigned quality score and topology loss score on the node, the scheduler directly reads the pre-calculated score when making a decision, avoiding the overhead of real-time topology calculation; Step S54: The scheduler reads and caches the static topology information once at startup, and subsequent scheduling decisions are read directly from the cache.

[0038] Topology scoring is a quantitative evaluation of the communication efficiency of a GPU combination. By using a space-for-time strategy, the calculation of GPU topology scores is separated from the scheduling path and completed in the node initialization stage. The results are cached in the node annotations. When the scheduler runs, it only needs to read the pre-calculated scores and does not need to repeat the complex combination evaluation, thereby reducing scheduling latency and improving scheduling throughput.

[0039] An AI computing resource management system, such as Figure 10As shown, the system specifically includes a hardware adaptation module, a resource allocation module, a quota scheduling module, an AI management module, a data storage module, and a sensory interaction module. The hardware adaptation module ensures compatibility with AI chips, shields differences in hardware drivers and instruction sets, and transforms physical resources into a unified, schedulable logical resource pool. It also unifies device plug-in interfaces, provides a unified API layer, handles instruction set differences between different chips, transforms physical GPUs or NPUs into schedulable logical units, and detects NVLink domains and topology. The resource allocation module uses queues as the basic unit for resource scheduling, setting different resource quotas in multi-level queues to achieve physical isolation for different environments, departments, and priorities. It uses queues as the basic unit to achieve resource isolation and allocation, forming a queue-resource binding relationship based on the established logical resource pool and queue configuration. The quota scheduling module performs fine-grained scheduling of resource allocation based on topology-aware scheduling and dynamic priority reduction. It executes refined scheduling decisions, forcibly allocating consecutively numbered devices based on topology-aware scheduling, reducing the priority of queues with high historical usage, and enabling high-priority tasks to preempt low-priority task resources. The AI ​​management module is used for personnel... The system implements logical isolation and access control for resources and data, and provides a unified database. The AI ​​management module achieves logical isolation and unified control of people, resources, and data through management oriented towards the AI ​​development process. It creates independent spaces for projects and teams, and provides unified storage and management of AI models, as well as unified storage and access to datasets. The data storage module is used to cache topology calculation data and task data. It provides high-performance data caching and storage for scheduling decisions and system operation, and is used to store pre-calculated GPU combination scores, cache NVLink domain and NUMA information, store task running status, queuing information, breakpoint data of training tasks, and persist scheduling records and monitoring data. The sensory interaction module is used to comprehensively monitor hardware, communication, and AI task parameters. The sensory interaction module, hardware adaptation module, resource allocation module, quota scheduling module, AI management module, and data storage module are electrically connected. The sensory interaction module provides the system's observability and human-computer interaction interface. It is responsible for collecting raw data from infrastructure and AI tasks, transforming the collected raw data into structured information that can be analyzed, and automatically correlates and analyzes cross-level abnormal indicators based on causal reasoning and knowledge graphs, providing a multi-dimensional monitoring view.

[0040] Principle of this invention: Based on a topology-aware scheduling method, node types are identified, and resources are allocated in segments according to priority. Guaranteed quotas are allocated to ensure basic operations, while quotas exceeding these guaranteed quotas are shared quotas. These shared quotas are allocated between adjacent priority segments using a segmented allocation approach. First, the basic quota for the corresponding priority is called. If the basic quota is insufficient for operations, the shared quota between the higher and next lower priority is called. If this still fails, the front-end scheduling unit preempts the basic quota of the next lower priority, ensuring tasks remain within the same high-speed interconnect domain. Fine-tuning of resource allocation is achieved, and the controller updates topology information in real time, caching the scheduler's allocated quality score, topology loss score, and static topology information. The scheduler directly reads pre-calculated scores and scheduling decisions, avoiding real-time topology computation overhead and improving real-time performance. Furthermore, after a task is preempted, some metadata is retained. Once the higher-priority task completes its computation, the preempted task is prioritized for recovery after the resources are released. When a task starts, it automatically resumes loading data from the last checkpoint before interruption, avoiding recalculation from the beginning, preventing wasted computing power, and improving computational efficiency.

[0041] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An AI computing resource management method, characterized in that, Includes the following steps: Step S1: Based on different environments and departments, deploy them in separate workspaces, set priorities, and adjust the priorities in real time according to historical data; Step S2: Receive and understand the natural language intent of the instructions through the LLM scheduler, dynamically adjust the strategy based on real-time feedback, verify the adjusted strategy, and filter the decision trajectory. Step S3: Allocate weight according to priority, allocate a basic quota to ensure basic operations, and allocate quotas exceeding the basic quota as shared quotas. The shared quotas are set between adjacent priority segments based on a segmented allocation method, serving as resources for allocation between priorities. Step S4: The scheduler identifies the node type and, according to the quota allocation method in step S3, calls the basic quota and shared quota of the corresponding priority, forcibly requiring the allocation of consecutively numbered devices to ensure that tasks are in the same high-speed interconnect domain. At the same time, when the quota of a high-priority computing task cannot meet the requirements, the low-priority computing task is temporarily interrupted, resources are released, the interrupted task is automatically saved and queued, waiting to resume execution, and when a low-priority task starts, data is automatically loaded from the latest checkpoint. Step S5: Construct a multi-layered topology based on the labels on the nodes. The controller automatically updates the members and structure to ensure that the topology information is consistent with the cluster state. At the same time, during the scheduling process, preprocessing replaces real-time topology calculation, allowing the scheduler to read the results directly from the cache. 2.The AI computing resource management method of claim 1, wherein, Setting the priority in step S1 includes the following steps: Step S11: Establish a unified priority hierarchy standard based on business scenarios, and create independent queues for different environments and departments. Each queue defines the resource quota that the workspace can obtain. Step S12: At the queue level, a preemption mechanism is used to ensure that high-priority tasks can obtain resources, and high-priority tasks can preempt resources from low-priority tasks. 3.The AI computing resource management method of claim 2, wherein, The priority adjustment in step S1 includes the following steps: Step S13: Track the historical resource usage of each queue through the scheduler and dynamically adjust the priority. When a queue uses more resources, its priority will be dynamically reduced when allocating shared resources in the future. Step S14: The longer the pause waiting time is, the higher the priority of the queue, and its priority rises to a level sufficient to obtain resources. 4.The AI computing resource management method of claim 3, wherein, The dynamic adjustment strategy in S2 includes the following steps: Step S21: The LLM scheduler automatically generates a queue list and arranges it in order by translating human-readable instructions into machine-executable scheduling policies; Step S22: Based on the queue list order in Step S21, when an error occurs in the previous queue, the LLM scheduler skips that queue and continues to try in the next queue using a binary backoff strategy: Step S23: Store failure mode and success strategy data in a vector database to provide reference data for subsequent scheduling.

5. The AI computing resource management method of claim 4, wherein, The resource quota allocation in step S3 includes the following steps: Step S31: Allocate resources proportionally according to priority, with higher priority tasks receiving more resources than lower priority tasks. Allocate basic quotas to maintain different priority tasks, and use quotas exceeding the sum of basic quotas for different priorities as shared quotas for different priority tasks. Step S32: The basic quotas of different priorities in step S31 are distributed in a segmented manner, and the shared quotas are distributed among different priorities according to the priority ratio. The basic quotas of different priorities and the allocated shared quotas are distributed at intervals. Step S33: When a high-priority task is being processed, the basic quota corresponding to the priority is called first. If the basic quota is insufficient for the operation, the shared quota between the high-priority task and the next lower priority task is called.

6. The AI computing resource management method of claim 5, wherein, The allocation of consecutive numbering devices in step S4 includes the following steps: Step S41: The scheduler obtains the hardware characteristics of the nodes through the node label or feature discovery mechanism. After identifying the node type, the scheduler calls the preset topology policy and uses the basic quota and shared quota of the corresponding priority. Step S42: Utilize the collaborative work of the scheduler and the mesh topology to continuously index in the physical loop, determine the physical location number of the GPU, determine the allocation combination under different GPU quantity requests based on hardware interconnect rules, index to complete contiguous blocks that meet the conditions, and the scheduler writes this allocation result into the annotation of the scheduling unit.

7. The AI computing resource management method of claim 6, wherein, The priority calculation in step S4 includes the following steps: Step S43: After a high-priority task is submitted, the scheduler detects high-priority computing resources. When the detected resources are insufficient, it simulates preemption and selects an eviction target, triggers a low-priority task to save a snapshot and evicts the low-priority task, and schedules the high-priority task. Step S44: Combining the resource reservation strategy, after a task is preempted, some metadata is retained. When the high-priority task in step S43 is completed, the preempted resources are released and the task is restored first. When the task starts, it automatically continues to load data from the last checkpoint before the interruption.

8. The AI ​​computing power resource management method according to claim 7, characterized in that, The multi-layer topology update in step S5 includes the following steps: Step S51: By defining the correspondence between topology levels and node labels in the controller, the controller will periodically scan the cluster nodes and automatically construct a multi-layered HyperNode topology based on the labels on the nodes. Step S52: When a node's label changes, or a node is added or removed, the controller automatically updates the HyperNode's members and structure to ensure that the topology information is always consistent with the cluster state and that the topology information is updated in real time.

9. The AI ​​computing power resource management method according to claim 8, characterized in that, The preprocessing in step S5 includes the following steps: Step S53: By pre-calculating the topology score of the GPU on the node and storing the assigned quality score and topology loss score on the node, the scheduler directly reads the pre-calculated score when making a decision, avoiding the overhead of real-time topology calculation; Step S54: The scheduler reads and caches the static topology information once at startup, and subsequent scheduling decisions are read directly from the cache.

10. An AI computing power resource management system, characterized in that, The system uses an AI computing resource management method as described in any one of claims 1-9, specifically including: The hardware adaptation module is used to ensure compatibility with AI chips, shield the differences in hardware drivers and instruction sets, and transform physical resources into a unified schedulable logical resource pool. The resource allocation module uses queues as the basic unit of resource scheduling, sets different resource quotas in multi-level queues, achieves physical isolation for different environments, departments, and sets queue priorities; The quota scheduling module performs fine-grained scheduling of resource allocation based on topology-aware scheduling and dynamic priority reduction. The AI ​​management module is used to logically isolate and control access to personnel, resources and data, and provides a unified database; A data storage module is provided for caching topology calculation data and task data. The sensory interaction module is used to comprehensively monitor hardware, communication, and AI task parameters. The sensory interaction module, hardware adaptation module, resource allocation module, quota scheduling module, AI management module, and data storage module are electrically connected.

Images