AI capacity expansion and contraction method and system for improving resource utilization rate

Abstract

The invention relates to the technical field of artificial intelligence, in particular to an AI capacity expansion and contraction method and system for improving the resource utilization rate, and the method comprises the following steps: node resource perception and classification modeling, AI task load clustering and reuse identification, and resource reuse scheduling strategy generation. The method has the beneficial effects that multitask multiplexing and dynamic merging are supported, and the overall resource utilization rate of the AI system is improved. The system has resource sensing capability and adapts to heterogeneous nodes and different load tasks. And the resource granularity is finely controlled, the redundancy capacity expansion is reduced, and the operation cost is reduced. And online learning and scheduling optimization are supported, and the elasticity and adaptability of the system are improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to an AI scaling method and system for improving resource utilization. Background Technology

[0002] In current large-scale deployments of AI systems, resource waste often occurs due to static resource configuration and low-load operation. For example, when the demand for AI inference services surges, it can easily trigger over-scaling, while when the demand subsides, it cannot scale down in time, ultimately resulting in an inefficient phenomenon of "resource peak-valley mismatch".

[0003] Existing scaling methods mostly rely on coarse-grained resource control and lack comprehensive analysis of task load volatility, node capability differences, and resource reuse scenarios. There is an urgent need to propose an AI scaling method that can dynamically adjust resource allocation and improve resource utilization. Summary of the Invention

[0004] The purpose of this invention is to provide an AI scaling method and system that improves resource utilization, so as to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: an AI scaling method for improving resource utilization, comprising the following steps:

[0006] Node resource awareness and classification modeling steps: Real-time collection of CPU, GPU, memory utilization and remaining capacity information of each computing node; Based on the pre-built capability model, classify the resource status of each computing node according to the collected resource information;

[0007] AI task load clustering and reuse identification steps: Analyze the resource usage patterns of the current AI inference / training tasks, identify tasks that can run concurrently through specific algorithms or rules; build a reuse scheduling model based on the identification results for subsequent resource scheduling reference;

[0008] Resource reuse scheduling strategy generation steps: Based on the remaining capacity of the node and the compatibility of task load, a refined resource scheduling strategy is generated using the reuse scheduling model to enable multiple tasks to share the same computing node or container resources.

[0009] Preferably, it also includes: a low-load merging and dynamic recycling mechanism step: continuously monitor the load of each computing node, and for computing nodes that have been in a low-load state for a long time, migrate the tasks running on the node to other suitable nodes; after the task migration is completed, release the redundant resources occupied by the long-term low-load node; for containers with high resource idle rate, perform a merge deployment operation to merge multiple low-load containers into one container to run tasks; or, when the resource idle rate reaches a certain threshold, perform a shrink operation on the container to reduce the resources it occupies.

[0010] Preferably, it also includes: a structural analysis and optimization feedback step: using graph structure feature analysis algorithms, including but not limited to degree centrality algorithms and bridge node identification algorithms, to analyze the topology structure composed of computing nodes and containers; based on the analysis results, providing a health score for the entire topology structure, which is used to measure the resource utilization efficiency and stability of the topology structure; and based on the health score and the graph structure feature analysis results, providing targeted optimization suggestions to improve the resource utilization of the topology structure.

[0011] Preferably, it also includes: a resource regulation feedback optimization step: recording the results of each resource scheduling execution, including but not limited to task allocation, resource usage, and task execution efficiency information; based on the recorded scheduling execution results, using reinforcement learning algorithms or feedback self-adjustment mechanisms to continuously optimize the resource allocation model; by continuously optimizing the resource allocation model, improving the resource utilization efficiency of the next round of resource scheduling strategy, and achieving more accurate and efficient resource allocation.

[0012] Preferably, the capability model is pre-built based on the hardware configuration of computing nodes, historical resource usage, and business demand factors; the reuse scheduling model is built based on task resource usage patterns, node resource status classification, and task compatibility rules; and the resource allocation model is built based on node resource information, task load information, and historical scheduling experience, and is continuously updated and improved in the resource regulation feedback optimization step.

[0013] A system for an AI scaling method to improve resource utilization includes:

[0014] Node resource awareness and classification modeling module: used to collect CPU, GPU, memory utilization and remaining capacity information of each computing node in real time; based on the preset capability model, classify the resource status of each computing node according to the collected resource information, and store the classification results in the system database;

[0015] AI task load clustering and reuse identification module: It is communicatively connected to the node resource perception and classification modeling module, and is used to analyze the resource usage pattern of the current AI inference / training task, identify tasks that can be run concurrently through a preset algorithm, construct a reuse scheduling model based on the identification results, and store the model data in the system database;

[0016] Resource reuse scheduling strategy generation module: It is communicatively connected to the node resource perception and classification modeling module and the AI ​​task load clustering and reuse identification module, respectively. It is used to generate a refined resource scheduling strategy based on the node's remaining capacity and task load compatibility using the reuse scheduling model, so as to enable multiple tasks to share the same computing node or container resources, and send the generated scheduling strategy to the execution module.

[0017] Preferably, it also includes: a low-load merging and dynamic reclamation module: which communicates with the node resource awareness and classification modeling module to continuously monitor the load of each computing node; for computing nodes that are in a low-load state for a long time, it migrates the tasks running on the node to other suitable nodes; after the task migration is completed, it releases the redundant resources occupied by the long-term low-load node and updates the resource release information to the system database; for containers with high resource idle rate, it performs a merge deployment operation to merge multiple low-load containers into one container to run tasks; or, when the resource idle rate reaches a preset threshold, it performs a shrink operation on the container to reduce the resources it occupies and updates the container adjustment information to the system database.

[0018] Preferably, it also includes: a structure analysis and optimization feedback module: communicating with the system database, using graph structure feature analysis algorithms, including but not limited to degree centrality algorithms and bridge node identification algorithms, to analyze the topology of computing nodes and containers; based on the analysis results, providing a health score for the entire topology, which is used to measure the resource utilization efficiency and stability of the topology, and storing the health score in the system database; based on the health score and the graph structure feature analysis results, providing targeted optimization suggestions, and displaying the optimization suggestions in a visual form to the system administrator or sending them directly to the relevant optimization execution module.

[0019] Preferably, it also includes: a resource regulation feedback optimization module: which is communicatively connected to the resource scheduling execution module, records the results of each resource scheduling execution, including but not limited to task allocation, resource usage, and task execution efficiency information, and stores the recorded information in the system database; based on the recorded scheduling execution results, it continuously optimizes the resource allocation model using reinforcement learning algorithms or feedback self-adjustment mechanisms; and updates the optimized resource allocation model to the system database for use by the subsequent resource scheduling strategy generation module, thereby improving the resource utilization efficiency of the next round of resource scheduling strategy.

[0020] Preferably, the capability model is pre-built based on the hardware configuration of computing nodes, historical resource usage, and business demand factors; the reuse scheduling model is built based on task resource usage patterns, node resource status classification, and task compatibility rules; the resource allocation model is built based on node resource information, task load information, and historical scheduling experience, and is continuously updated and improved in the resource regulation feedback optimization module; the modules interact and collaborate with each other through a system bus or network communication protocol.

[0021] Compared with the prior art, the beneficial effects of the present invention are:

[0022] This invention proposes an AI scaling method and system to improve resource utilization, supporting multi-task reuse and dynamic merging to enhance the overall resource utilization of the AI ​​system. It possesses resource awareness capabilities, adapting to heterogeneous nodes and different workloads. It provides fine-grained control over resource granularity, reducing redundant scaling and lowering operating costs. It supports online learning and scheduling optimization, improving system resilience and adaptability. Attached Figure Description

[0023] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the present invention clear and complete, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only some, not all, embodiments of the present invention, and are merely illustrative of the embodiments of the present invention. They are not intended to limit the embodiments of the present invention. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] Example 1, please refer to Figure 1 This invention provides a technical solution: an AI scaling method to improve resource utilization, comprising the following steps:

[0026] 1. Node resource awareness and classification modeling: Real-time collection of CPU, GPU, memory utilization and remaining capacity of each computing node, and classification of node resource status based on capability model.

[0027] 2. AI Task Load Clustering and Reuse Identification: Analyze the resource usage patterns of current AI inference / training tasks, identify tasks that can run concurrently, and build a reuse scheduling model.

[0028] 3. Resource reuse scheduling strategy generation: Based on the remaining capacity of nodes and the compatibility of task load, a fine-grained resource scheduling strategy is generated to enable multiple tasks to share the same computing node or container resources.

[0029] 4. Low-load merging and dynamic recycling mechanism: For nodes with low load for a long time, perform task migration and release redundant resources; for containers with high resource idle rate, perform merge deployment or scaling down operation.

[0030] 5. Structural Analysis and Optimization Feedback: Combining graph structure feature analysis algorithms (such as degree centrality and bridge node identification), it provides topology health scores and optimization suggestions.

[0031] 6. Resource regulation and feedback optimization: Record the scheduling execution results and continuously optimize the resource allocation model based on reinforcement learning or feedback self-adjustment mechanism to improve the resource utilization efficiency of the next round of scheduling strategy.

[0032] Example 2, based on Example 1, proposes a system for an AI scaling method to improve resource utilization, comprising:

[0033] Node resource awareness and classification modeling module: used to collect CPU, GPU, memory utilization and remaining capacity information of each computing node in real time; based on the preset capability model, classify the resource status of each computing node according to the collected resource information, and store the classification results in the system database;

[0034] AI task load clustering and reuse identification module: It is communicatively connected to the node resource perception and classification modeling module, and is used to analyze the resource usage pattern of the current AI inference / training task, identify tasks that can be run concurrently through a preset algorithm, construct a reuse scheduling model based on the identification results, and store the model data in the system database;

[0035] Resource reuse scheduling strategy generation module: It is communicatively connected to the node resource perception and classification modeling module and the AI ​​task load clustering and reuse identification module, respectively. It is used to generate a refined resource scheduling strategy based on the node's remaining capacity and task load compatibility using the reuse scheduling model, so as to enable multiple tasks to share the same computing node or container resources, and send the generated scheduling strategy to the execution module.

[0036] It also includes: a low-load merging and dynamic reclamation module: communicating with the node resource awareness and classification modeling module, continuously monitoring the load of each computing node; for computing nodes that have been in a low-load state for a long time, migrating the tasks running on the node to other suitable nodes; after the task migration is completed, releasing the redundant resources occupied by the long-term low-load node and updating the resource release information to the system database; for containers with high resource idle rates, performing a merge deployment operation to merge multiple low-load containers into one container to run tasks; or, when the resource idle rate reaches a preset threshold, performing a shrink operation on the container to reduce the resources it occupies and updating the container adjustment information to the system database;

[0037] It also includes: a structural analysis and optimization feedback module: which communicates with the system database and uses graph structure feature analysis algorithms, including but not limited to degree centrality algorithms and bridge node identification algorithms, to analyze the topology of computing nodes and containers; based on the analysis results, it provides a health score for the entire topology, which measures the resource utilization efficiency and stability of the topology, and stores the health score in the system database; based on the health score and the graph structure feature analysis results, it provides targeted optimization suggestions and displays the optimization suggestions in a visual form to the system administrator or sends them directly to the relevant optimization execution module;

[0038] It also includes: a resource regulation feedback optimization module: which communicates with the resource scheduling execution module, records the results of each resource scheduling execution, including but not limited to task allocation, resource usage, and task execution efficiency information, and stores the recorded information in the system database; based on the recorded scheduling execution results, it uses reinforcement learning algorithms or feedback self-adjustment mechanisms to continuously optimize the resource allocation model; and updates the optimized resource allocation model to the system database for use by the subsequent resource scheduling strategy generation module to improve the resource utilization efficiency of the next round of resource scheduling strategy.

[0039] The capability model is pre-built based on the hardware configuration of computing nodes, historical resource usage, and business demand factors; the reuse scheduling model is built based on task resource usage patterns, node resource status classification, and task compatibility rules; the resource allocation model is built based on node resource information, task load information, and historical scheduling experience, and is continuously updated and improved in the resource regulation feedback optimization module; the modules interact and collaborate with each other through system bus or network communication protocols.

[0040] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for scaling up and down AI to improve resource utilization, characterized in that: Includes the following steps: Node resource awareness and classification modeling steps: Real-time collection of CPU, GPU, memory utilization and remaining capacity information of each computing node; Based on the pre-built capability model, classify the resource status of each computing node according to the collected resource information; AI task load clustering and reuse identification steps: Analyze the resource usage patterns of the current AI inference / training tasks, identify tasks that can run concurrently through specific algorithms or rules; build a reuse scheduling model based on the identification results for subsequent resource scheduling reference; Resource reuse scheduling strategy generation steps: Based on the remaining capacity of the node and the compatibility of task load, a refined resource scheduling strategy is generated using the reuse scheduling model to enable multiple tasks to share the same computing node or container resources.

2. The AI ​​scaling method for improving resource utilization according to claim 1, characterized in that: Also includes: The low-load merging and dynamic recycling mechanism involves the following steps: Continuously monitor the load of each compute node; for compute nodes that have been in a low-load state for a long time, migrate the tasks running on that node to other suitable nodes; after the task migration is completed, release the redundant resources occupied by the long-term low-load node; for containers with high resource idle rates, perform a merge deployment operation to merge multiple low-load containers into one container to run tasks; or, when the resource idle rate reaches a certain threshold, perform a scaling-down operation on the container to reduce the resources it occupies.

3. The AI ​​scaling method for improving resource utilization according to claim 2, characterized in that: Also includes: Structural analysis and optimization feedback steps: Graph structure feature analysis algorithms, including but not limited to degree centrality algorithms and bridge node identification algorithms, are used to analyze the topology composed of computing nodes and containers; based on the analysis results, a health score is provided for the entire topology, which is used to measure the resource utilization efficiency and stability of the topology; based on the health score and the graph structure feature analysis results, targeted optimization suggestions are given to improve the resource utilization of the topology.

4. The AI ​​scaling method for improving resource utilization according to claim 3, characterized in that: Also includes: Resource regulation feedback optimization steps: Record the results of each resource scheduling execution, including but not limited to task allocation, resource usage, and task execution efficiency information; based on the recorded scheduling execution results, use reinforcement learning algorithms or feedback self-adjustment mechanisms to continuously optimize the resource allocation model; by continuously optimizing the resource allocation model, improve the resource utilization efficiency of the next round of resource scheduling strategy, and achieve more accurate and efficient resource allocation.

5. The AI ​​scaling method for improving resource utilization according to claim 4, characterized in that: The capability model is pre-built based on the hardware configuration of computing nodes, historical resource usage, and business demand factors; the reuse scheduling model is built based on task resource usage patterns, node resource status classification, and task compatibility rules; the resource allocation model is built based on node resource information, task load information, and historical scheduling experience, and is continuously updated and improved in the resource regulation feedback optimization step.

6. A system for the AI ​​scaling method to improve resource utilization according to claim 5, characterized in that: include: Node resource awareness and classification modeling module: used to collect CPU, GPU, memory utilization and remaining capacity information of each computing node in real time; Based on the preset capability model, the resource status of each computing node is classified according to the collected resource information, and the classification results are stored in the system database. AI task load clustering and reuse identification module: It is communicatively connected to the node resource perception and classification modeling module, and is used to analyze the resource usage pattern of the current AI inference / training task, identify tasks that can be run concurrently through a preset algorithm, construct a reuse scheduling model based on the identification results, and store the model data in the system database; The resource reuse scheduling strategy generation module is communicatively connected to the node resource perception and classification modeling module and the AI ​​task load clustering and reuse identification module, respectively. It is used to generate a refined resource scheduling strategy based on the node's remaining capacity and task load compatibility using the reuse scheduling model, so as to enable multiple tasks to share the same computing node or container resources, and send the generated scheduling strategy to the execution module.

7. The system according to claim 6, characterized in that: Also includes: Low-load merging and dynamic reclamation module: Communicates with the node resource awareness and classification modeling module to continuously monitor the load of each computing node. For computing nodes that are in a low-load state for a long time, it migrates the tasks running on the node to other suitable nodes. After the task migration is completed, it releases the redundant resources occupied by the long-term low-load node and updates the resource release information to the system database. For containers with high resource idle rates, it performs a merge deployment operation, merging multiple low-load containers into one container to run tasks. Alternatively, when the resource idle rate reaches a preset threshold, it performs a scaling-down operation on the container to reduce its resource consumption and updates the container adjustment information to the system database.

8. The system according to claim 7, characterized in that: Also includes: The structural analysis and optimization feedback module communicates with the system database and uses graph structure feature analysis algorithms, including but not limited to degree centrality algorithms and bridge node identification algorithms, to analyze the topology of computing nodes and containers. Based on the analysis results, it provides a health score for the entire topology, which measures the resource utilization efficiency and stability of the topology, and stores the health score in the system database. Based on the health score and graph structure feature analysis results, it provides targeted optimization suggestions and displays these suggestions in a visual form to the system administrator or sends them directly to the relevant optimization execution module.

9. A system according to claim 8, characterized in that: Also includes: Resource regulation feedback optimization module: Communicates with the resource scheduling execution module, records the results of each resource scheduling execution, including but not limited to task allocation, resource usage, and task execution efficiency information, and stores the recorded information in the system database; based on the recorded scheduling execution results, it continuously optimizes the resource allocation model using reinforcement learning algorithms or feedback self-adjustment mechanisms; and updates the optimized resource allocation model to the system database for use by the subsequent resource scheduling strategy generation module, improving the resource utilization efficiency of the next round of resource scheduling strategies.

10. A system according to claim 9, characterized in that: The capability model is pre-built based on the hardware configuration of computing nodes, historical resource usage, and business demand factors; the reuse scheduling model is built based on task resource usage patterns, node resource status classification, and task compatibility rules; the resource allocation model is built based on node resource information, task load information, and historical scheduling experience, and is continuously updated and improved in the resource regulation feedback optimization module; the modules interact and collaborate with each other through system bus or network communication protocols.

Images