Computing power self-adaptive scheduling method, device and equipment of AI server cluster and medium
By optimizing the allocation of computing resources through multi-scale spatiotemporal attention prediction and hypergraph models, the adaptive scheduling problem of AI server clusters is solved, improving resource utilization and computing efficiency while reducing latency and communication costs.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHENZHEN YUSHENG IND CO LTD
- Filing Date
- 2026-02-11
- Publication Date
- 2026-05-08
AI Technical Summary
In existing technologies, the computing resources of AI servers cannot be adaptively adjusted, resulting in long cold migration interruption times, affecting business continuity. Furthermore, hot migration algorithms fail to effectively utilize the characteristics of the SR-IOV scenario, causing resource waste and imbalance.
By collecting multi-dimensional resource status and workload data of AI server clusters in real time, and using multi-scale spatiotemporal attention prediction models and multi-objective constraint reinforcement learning algorithms, an adaptive scheduling strategy is generated. Combined with a hypergraph model, refined resource allocation is performed to optimize the allocation of computing power tasks.
It enables adaptive adjustment of the computing power of the AI server cluster, improves resource utilization, reduces cross-node communication costs, ensures that the response latency of high-priority tasks is within a reasonable range, and improves computing efficiency and resource utilization.
Smart Images
Figure CN121996427A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of communication technology, and more specifically, to a method, apparatus, device, and medium for adaptive scheduling of computing power in an AI server cluster. Background Technology
[0002] With the widespread application and promotion of artificial intelligence and large-scale models, the demand for AI computing resources is increasing exponentially. Currently, the main demand comes from AI servers in intelligent computing centers. These AI servers utilize multiple GPU cards to provide AI computing power, and the AI operations in data centers are deployed on these GPU cards within the AI server resource pool. Currently, AI servers primarily use GPUs to execute AI computing tasks, while CPUs handle data input and AI program execution. In intelligent computing centers, GPUs are the primary AI computing power, scheduled and managed through a cloud platform. To improve the management efficiency and utilization of computing resources, servers now require virtualization technology and cloud platforms for scheduling and management of computing resources.
[0003] In related technologies, GPU cards utilize SR-IOV virtualization technology to virtualize the Physical Function (PF) into multiple Virtual Function (VF) devices for use by virtual machines, thereby improving GPU computing power utilization. However, currently, most SR-IOV scenarios employ cold migration technology, which results in prolonged interruptions. For scenarios requiring continuous operation, such as training and inference, this significantly impacts business operations and leads to wasted and unbalanced GPU computing resources. Furthermore, the computing power scheduling algorithms used in other scenarios' hot migration techniques, due to insufficient consideration of the characteristics of the SR-IOV scenario, cannot determine the optimal target computing node for hot migration in the SR-IOV environment. Summary of the Invention
[0004] The purpose of this disclosure is to provide a method, apparatus, device, and medium for adaptive scheduling of computing power in AI server clusters, so as to solve the technical problem that server computing power cannot be adaptively adjusted.
[0005] To achieve the above objectives, the first aspect of this disclosure provides a computing power adaptive scheduling method for an AI server cluster, the method comprising: Real-time collection of multi-dimensional resource status data and workload data corresponding to the AI server cluster. The multi-dimensional resource status data is used to indicate whether the multi-dimensional resources corresponding to each AI server are currently in an enabled or dormant state. The workload data is used to indicate the overall load of the multi-dimensional resources corresponding to each AI server. Based on the workload data and the historical load data of each AI server, the short-term and long-term workloads of each AI server are dynamically predicted using a multi-scale spatiotemporal attention prediction model, generating predicted load data for each AI server. Based on the predicted load data, the multi-dimensional resource status data, and the workload data, the computing power of each AI server cluster is adaptively adjusted through a reinforcement learning algorithm with multi-objective constraints to generate an initial scheduling strategy for the current computing power task of the AI server cluster. The multi-objective constraints include at least one of the following: load balancing constraints, communication cost constraints, and task resource requirement constraints. The resource allocation optimization algorithm based on the hypergraph model finely adjusts the initial scheduling strategy to generate the target resource allocation strategy for the AI server cluster. The computing power tasks are then scheduled and allocated using the target resource allocation strategy to generate a computing power task allocation scheme for each AI server. In the hypergraph model, a hypernode represents an AI server node, and a hyperedge represents an AI computing task and connects all AI server nodes that meet the resource requirements of that task.
[0006] Optionally, in some embodiments, the step of dynamically predicting the short-term and long-term workloads of each AI server based on the workload data and the historical load data of each AI server using a multi-scale spatiotemporal attention prediction model, and generating predicted load data corresponding to each AI server, includes: The historical load data is decomposed to generate trend terms, periodic terms and residual terms corresponding to the historical load data. Multinomial regression, Fourier series decomposition and long short-term memory network are used to model the cluster physical topology and network connection relationship model, wherein the cluster physical topology and network connection relationship model is a graph model. The graph convolutional network is used to capture the relationships between nodes in the cluster physical topology and network connection model, so as to generate the load relationship and propagation characteristic relationship between the nodes. Based on the load correlation relationship and the propagation feature relationship, the temporal features and topology correlation features in the cluster physical topology and network connection relationship model are fused through a spatiotemporal attention mechanism to generate multi-step predicted load results for the future load data of each AI server. Based on the multi-step predicted load results, the predicted load data for each AI server is determined by an adaptive selector based on ensemble learning.
[0007] Optionally, in some embodiments, the step of capturing the associations between nodes in the cluster physical topology and network connection relationship model through a graph convolutional network to generate load associations and propagation characteristic relationships between nodes includes: Obtain the neighboring nodes between each node in the cluster physical topology and network connection relationship model, as well as the connection relationship between each node and its neighboring nodes; Based on the adjacent nodes and the connection relationships, the load association relationship and the propagation characteristic relationship between each node are determined.
[0008] Optionally, in some embodiments, the resource allocation optimization algorithm based on the hypergraph model finely adjusts the initial scheduling strategy to generate a target resource allocation strategy for the AI server cluster, including: Obtain the hypergraph model, wherein the hypernodes in the hypergraph model represent each AI server node, the hypergraph model includes a multi-dimensional resource attribute vector, the hyperedges represent AI computing tasks, and connect all AI server nodes that can meet the resource requirements of the task. The resource allocation problem is formalized as a constrained hypergraph balancing partitioning problem. The optimization objective is to minimize the cross-node communication cost while satisfying the task resource requirements and maintaining load balancing among nodes, so as to generate a hypergraph partitioning algorithm to be solved. Based on the initial scheduling strategy, an iteratively improved hierarchical approach is used to solve the hypergraph partitioning algorithm to generate the optimal partitioning method corresponding to the hypergraph model. Based on the optimal partitioning method and the corresponding set of AI server nodes, the target resource allocation strategy for the AI server cluster is generated.
[0009] Optionally, in some embodiments, the method further includes: Based on the target resource allocation strategy, the multi-scale spatiotemporal attention prediction model and the reinforcement learning algorithm for multi-objective constraints are optimized to generate a target multi-scale spatiotemporal attention prediction model and a target multi-objective constraint reinforcement learning algorithm.
[0010] Optionally, in some embodiments, the step of scheduling and allocating the computing power tasks according to the target resource allocation strategy to generate a computing power task allocation scheme corresponding to each AI server includes: Obtain real-time data metrics for each AI server; Based on the target resource allocation strategy and the real-time data indicators, the computing power of each AI server is adjusted by horizontally expanding the number of replicas or vertically adjusting the resource limit. Based on the computing power task and the horizontal replica number expansion or vertical resource limit adjustment, determine the AI server nodes that need to be added or removed in the AI server cluster, as well as the computing power tasks that need to be split. Based on the computing power tasks that need to be split and the AI server nodes that need to be added or removed, a computing power task allocation scheme is generated for each AI server. Optionally, in some embodiments, the step of scheduling and allocating the computing power tasks according to the target resource allocation strategy to generate a computing power task allocation scheme corresponding to each AI server includes: Obtain the graded service quality requirements corresponding to the computing power task; The computing power tasks are scheduled and allocated using the target resource allocation strategy to generate an initial computing power task allocation scheme; Based on the tiered service quality requirements, the initial computing power task allocation scheme is adjusted to generate the computing power task allocation scheme corresponding to each AI server.
[0011] According to a second aspect of this disclosure, a computing power adaptive scheduling device for an AI server cluster is provided, the device comprising: The acquisition module is used to collect multi-dimensional resource status data and workload data corresponding to the AI server cluster in real time. The multi-dimensional resource status data is used to indicate whether the multi-dimensional resources corresponding to each AI server are currently in an enabled or dormant state. The workload data is used to indicate the overall load of the multi-dimensional resources corresponding to each AI server. The first generation module is used to dynamically predict the short-term and long-term workloads of each AI server based on the workload data and the historical load data of each AI server, using a multi-scale spatiotemporal attention prediction model, and generate predicted load data for each AI server respectively. The second generation module is used to adaptively adjust the computing power of each AI server cluster based on the predicted load data, the multi-dimensional resource status data, and the workload data through a reinforcement learning algorithm with multi-objective constraints, and generate an initial scheduling strategy for the AI server cluster corresponding to the current computing power task. The multi-objective constraints include at least one of the following: load balancing constraints, communication cost constraints, and task resource requirement constraints. The execution module is used to fine-tune the initial scheduling strategy based on the resource allocation optimization algorithm of the hypergraph model, generate the target resource allocation strategy of the AI server cluster, and schedule and allocate the computing power tasks through the target resource allocation strategy to generate the computing power task allocation scheme corresponding to each AI server. In the hypergraph model, the hypernode represents the AI server node, and the hyperedge represents the AI computing task and connects all AI server nodes that meet the resource requirements of the task.
[0012] According to a third aspect of this disclosure, an electronic device is provided, comprising: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the method according to any one of the first aspects of this disclosure.
[0013] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, characterized in that, when executed by a processor, the program implements the steps of the method described in any of the second aspects of this disclosure.
[0014] Through the above technical solution, multi-dimensional resource status data and workload data corresponding to the AI server cluster are collected in real time. The multi-dimensional resource status data indicates whether the multi-dimensional resources of each AI server are currently in an active or dormant state, and the workload data indicates the overall load of the multi-dimensional resources of each AI server. Based on the workload data and the historical load data of each AI server, a multi-scale spatiotemporal attention prediction model is used to dynamically predict the short-term and long-term workloads of each AI server, generating predicted load data for each AI server. Based on the predicted load data, multi-dimensional resource status data, and workload data, a multi-objective constraint reinforcement learning algorithm is used to adaptively adjust the computing power of each AI server cluster, generating an initial scheduling strategy for the current computing power tasks of the AI server cluster. A resource allocation optimization algorithm based on a hypergraph model is used to fine-tune the initial scheduling strategy, generating a target resource allocation strategy for the AI server cluster. The computing power tasks are then scheduled and allocated according to the target resource allocation strategy, generating a computing power task allocation scheme for each AI server. This enables adaptive adjustment of the computing power of the AI server cluster. Based on the relationship between the current computing power and load of different AI servers, the computing power task allocation scheme of different AI servers can be dynamically adjusted, making full use of the computing power resources of each AI server and improving the task computing efficiency of the AI server cluster.
[0015] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description
[0016] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the following detailed description to explain the present disclosure, but do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart illustrating an adaptive scheduling method for computing power of an AI server cluster according to an exemplary embodiment.
[0017] Figure 2 This is a block diagram illustrating a computing power adaptive scheduling device for an AI server cluster according to an exemplary embodiment.
[0018] Figure 3 This is a block diagram illustrating an electronic device 300 according to an exemplary embodiment. Detailed Implementation
[0019] The specific embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit this disclosure.
[0020] This application aims to build a next-generation intelligent computing scheduling system to address the core pain points of current AI server clusters, such as resource fragmentation, mixed workloads, and delayed scheduling response. The system will be designed with intelligent perception, dynamic allocation, and elastic scaling in mind, achieving three core objectives: increasing the average utilization rate of expensive computing resources such as GPUs from the industry average of 35% to over 65%; ensuring that the P99 latency of high-priority inference services remains stable within 80 milliseconds under mixed load scenarios; and reducing overall operating costs by 25% through fine-grained scheduling and resource integration.
[0021] Figure 1 This is a flowchart illustrating an adaptive scheduling method for computing power in an AI server cluster, according to an exemplary embodiment. Figure 1 As shown, the method includes at least the following steps.
[0022] Step S101: Collect multi-dimensional resource status data and workload data corresponding to the AI server cluster in real time. The multi-dimensional resource status data is used to indicate whether the multi-dimensional resources corresponding to each AI server are currently in an enabled or dormant state, and the workload data is used to indicate the overall load of the multi-dimensional resources corresponding to each AI server.
[0023] For example, this embodiment adopts a hybrid architecture that combines centralized coordination with distributed execution, which ensures global resource optimization and adapts to the special needs of various workloads.
[0024] Central Scheduling Brain (Global Optimization Layer): As the system's decision-making hub, it maintains a comprehensive view of cluster resources and workload status. This layer is built on an enhanced Kubernetes scheduling framework but deeply integrates multi-objective optimization algorithms and predictive decision-making models. It does not directly intervene in real-time task scheduling, but rather formulates resource allocation strategies, handles long-cycle task placement, and resolves macro-level issues such as cross-node resource coordination.
[0025] Distributed scheduling unit (local execution layer): Deploy specialized scheduling components for different types of workload characteristics: Inference Service Scheduler: Based on a service mesh architecture, it focuses on millisecond-level response and fast elastic scaling, and adopts a scaling strategy that combines reactive and predictive approaches.
[0026] Training task scheduler: designed for long-running, computationally intensive jobs, supporting checkpoint / recovery mechanisms, topology-aware scheduling, and resource preemption.
[0027] Interactive Environment Scheduler: Provides data scientists and researchers with a fast-response development environment that supports on-demand creation and dynamic resource adjustment.
[0028] Intelligent Coordination Plane (Perception and Optimization Layer): This is the system's "nervous system," containing three core functional modules: Full-domain monitoring and perception module: Constructs a real-time digital twin of the cluster status through high-frequency data acquisition (second-level).
[0029] Multidimensional analysis and decision-making module: integrates rule engine, prediction model and optimization algorithm to generate scheduling strategy.
[0030] Strategy distribution and execution module: Transforms optimization strategies into specific scheduling instructions and monitors the execution effect.
[0031] The system is divided into four layers from bottom to top: Infrastructure Abstraction Layer: Unifies the encapsulation of heterogeneous computing resources (GPU / TPU / ASIC), high-speed networks (InfiniBand / RoCE), and storage systems, providing standardized resource interfaces.
[0032] Resource Management Layer: Enables virtualization, isolation, and quota management of physical resources, and supports various GPU virtualization technologies (MIG, vGPU, time-slice sharing).
[0033] Scheduling decision layer: includes intelligent components such as core scheduling algorithms, elastic scaling controllers, and interference sensing modules.
[0034] Service Interface Layer: Provides a unified user interface and API for different users and applications.
[0035] Step S102: Based on the workload data and the historical load data of each AI server, the short-term and long-term workloads of each AI server are dynamically predicted using a multi-scale spatiotemporal attention prediction model, generating the predicted load number for each AI server.
[0036] For example, in this embodiment, a multi-scale spatiotemporal attention prediction model is designed to address the nonlinear and periodic characteristics of AI workloads. The core innovation of this model lies in the deep fusion of the periodic features of the time dimension and the correlation features of the spatial dimension.
[0037] The time dimension employs an enhanced time-series decomposition architecture, decomposing the load sequence into trend, period, and residual components, which are then modeled separately. The trend term is fitted using multinomial regression to account for long-term changes. The periodic terms are processed by Fourier series decomposition to handle the superposition of multiple periods; The residual terms utilize an LSTM network to capture nonlinear patterns.
[0038] A spatial dimension is used to construct a cluster topology graph attention network, modeling the server nodes and their connections in the cluster as a graph structure. A graph convolutional network is then used to capture load propagation and mutual influence between nodes. For nodes that are physically close or have tight network connections, the model learns the load correlations between them, thus more accurately predicting local load fluctuations.
[0039] The system maintains a pool of prediction models, which includes: Statistical model: Seasonal Autoregressive Moving Average (SARIMA) model, suitable for stable cyclical loads. Classic machine learning model: Gradient Boosting Decision Tree (GBDT), which requires high feature engineering skills but offers strong interpretability. Deep learning model: Transformer-based time series forecasting model, which excels at capturing long-term dependencies; The system dynamically evaluates the performance of each model under the current load mode using an online learning selector, and generates the final prediction using a weighted voting mechanism. This design ensures both prediction accuracy and enhances the system's robustness.
[0040] For example, in this embodiment, the load prediction accuracy is improved by 15%-20% compared with the existing LSTM model through the multi-scale spatiotemporal attention prediction model; the cross-node communication cost is reduced by more than 30% through the hypergraph partitioning algorithm, and the cluster resource utilization rate is increased from the industry average of 35% to more than 65%; in mixed load scenarios, the P99 latency of high-priority inference tasks is stabilized within 80 milliseconds.
[0041] Optionally, in some embodiments, step S102 above includes: Historical load data is decomposed to generate trend, periodic and residual terms. Multinomial regression, Fourier series decomposition and long short-term memory network are used to model the cluster physical topology and network connection relationship model, which is a graph model. The graph convolutional network is used to capture the relationships between nodes in the cluster physical topology and network connection model, so as to generate the load relationship and propagation characteristic relationship between the nodes. Based on the load correlation and propagation characteristic relationship, the temporal features and topology correlation features in the cluster physical topology and network connection relationship model are fused through the spatiotemporal attention mechanism to generate multi-step predicted load results for the future load data of each AI server. Based on the multi-step predicted load results, the predicted load data for each AI server is determined by an adaptive selector based on ensemble learning.
[0042] Optionally, in some embodiments, the above step: "capturing the relationships between nodes in the cluster physical topology and network connectivity model using a graph convolutional network to generate load relationships and propagation characteristic relationships between nodes" includes: Obtain the neighboring nodes between each node in the cluster physical topology and network connection relationship model, as well as the connection relationship between each node and its neighboring nodes; Based on adjacent nodes and connection relationships, determine the load association and propagation characteristic relationships between each node.
[0043] Step S103: Based on the predicted load data, multi-dimensional resource status data, and workload data, the computing power of each AI server cluster is adaptively adjusted using a multi-objective constraint reinforcement learning algorithm to generate an initial scheduling strategy for the current computing power task of the AI server cluster.
[0044] In some embodiments, the multi-objective constraint reinforcement learning algorithm is used to indicate the computing power objective of the AI server. For example, the multi-objective constraint may include at least one of the following: minimizing cross-node communication cost, maximizing cluster resource utilization, ensuring load balancing between nodes, and meeting task service quality (QoS) requirements.
[0045] Step S104: Based on the hypergraph model, the resource allocation optimization algorithm finely adjusts the initial scheduling strategy to generate the target resource allocation strategy for the AI server cluster. The computing power tasks are then scheduled and allocated according to the target resource allocation strategy to generate the computing power task allocation scheme for each AI server.
[0046] In some embodiments, in the hypergraph model, a hypernode represents an AI server node, and a hyperedge represents an AI computing task and connects all AI server nodes that meet the resource requirements of that task.
[0047] For example, in this embodiment, the cluster scheduling problem is formalized as a partially observable Markov decision process. The state space is designed as a multidimensional hybrid vector, including: Cluster global status: resource utilization matrix in various dimensions, task queue status, network topology status; Workload characteristics: task type distribution, resource requirement patterns, priority, and time constraints; Environmental context: time period information, historical performance data, cost constraint parameters.
[0048] The action space is designed as a hierarchical decision-making structure: the top-level decision selects the scheduling strategy type (such as prioritizing latency, prioritizing throughput, cost optimization, etc.), and the bottom-level decision executes specific resource allocation and task scheduling under the selected strategy.
[0049] A hierarchical reinforcement learning architecture is adopted, with high-level policies learning macro-level scheduling strategy selection and low-level policies learning specific scheduling operations. The training process consists of two phases: Offline pre-training phase: Policy initialization is performed using historical scheduling data and behavior cloning techniques, and then further optimized in a simulated environment using offline reinforcement learning algorithms (such as conservative Q-learning).
[0050] Online fine-tuning phase: After system deployment, online learning is conducted within strict security boundaries. Importance sampling and constraint policy optimization techniques are employed to ensure that new policies do not significantly deviate from validated security policies. Simultaneously, a robust rollback mechanism is established to automatically switch to a backup rule scheduler when performance degradation exceeds a threshold.
[0051] In some embodiments, the state space includes a cluster resource utilization matrix, task queue length, and node load variance; the action space includes resource allocation ratio adjustment, task migration decision, and node scaling up / down operation; the reward function is a weighted summation function: reward value = 0.4 × (1 - cross-node communication cost ratio) + 0.3 × (1 - node load variance) + 0.3 × task on-time completion rate.
[0052] Optionally, in some embodiments, step S104 above includes: Obtain the hypergraph model, where the hypernodes in the hypergraph model represent each AI server node, the hypergraph model includes a multi-dimensional resource attribute vector, the hyperedges represent AI computing tasks, and connect all AI server nodes that can meet the resource requirements of the task. The resource allocation problem is formalized as a constrained hypergraph balancing partitioning problem. The optimization objective is to minimize the cross-node communication cost while satisfying the task resource requirements and maintaining load balancing among nodes, so as to generate a hypergraph partitioning algorithm to be solved. Based on the initial scheduling strategy, an iteratively improved hierarchical approach is adopted to solve the hypergraph partitioning algorithm to generate the optimal partitioning method corresponding to the hypergraph model. Based on the optimal partitioning method and the corresponding set of AI server nodes, a target resource allocation strategy for the AI server cluster is generated.
[0053] For example, in this embodiment, traditional graph models are insufficient to represent the complex multi-resource constraints of AI tasks. Therefore, this embodiment introduces hypergraph theory to model the scheduling problem. In the hypergraph: A supernode represents a server node and contains a multidimensional resource attribute vector. The hyperedge represents the resource requirements of a task and connects all server nodes that satisfy the resource constraints of that task. The advantage of this modeling method is that it can naturally express the complex constraint that "a task requires multiple resource combinations", and it can find an optimal task placement scheme through hypergraph segmentation algorithm.
[0054] The resource allocation problem is transformed into a constrained hypergraph balancing partitioning problem. The algorithm aims to minimize cross-server communication costs and maintain load balancing across servers while meeting task resource requirements.
[0055] This application proposes an iteratively improved hierarchical hypergraph partitioning algorithm: Coarsening stage: Through hyperedge shrinkage technology, large-scale hypergraphs are gradually simplified into small-scale hypergraphs; Initial partitioning phase: Initial partitions are generated using heuristic algorithms on small-scale hypergraphs; Refinement stage: Through multi-level optimization techniques (such as hypergraph expansion using the Fiduccia-Mattheyses algorithm), the partitioning quality is gradually optimized; Projection phase: Project the optimized partitions of the small-scale hypergraph back to the original hypergraph; The algorithm takes into account the resource constraints unique to AI, such as GPU memory, video memory bandwidth, and NVLink connection, and can generate scheduling schemes with a quality far exceeding that of traditional bin packing algorithms.
[0056] Optionally, in some embodiments, after step S104 above, the method further includes: Based on the target resource allocation strategy, the multi-scale spatiotemporal attention prediction model and the reinforcement learning algorithm with multi-objective constraints are optimized to generate the target multi-scale spatiotemporal attention prediction model and the target multi-objective constraint reinforcement learning algorithm.
[0057] For example, in this embodiment, the fusion weights of the multi-scale spatiotemporal attention prediction model and the reward function coefficients of the reinforcement learning algorithm are dynamically adjusted based on the execution effect of the target resource allocation strategy (such as the error between the predicted load and the actual load, and the completion efficiency of the scheduling task), so as to realize online iterative optimization of the model and the algorithm.
[0058] Optionally, in some embodiments, step S104 above includes: Obtain real-time data metrics for each AI server; Based on the target resource allocation strategy and real-time data indicators, the computing power of each AI server is adjusted by horizontally expanding the number of replicas or vertically adjusting the resource limits. Based on the computing power tasks and the expansion of horizontal replica counts or the adjustment of vertical resource limits, determine the AI server nodes that need to be added or removed in the AI server cluster, as well as the computing power tasks that need to be split. Based on the required split computing power tasks and the AI server nodes that need to be added or removed, generate a computing power task allocation scheme for each AI server.
[0059] In some embodiments, horizontal replica expansion refers to expanding the overall computing power of the cluster by increasing the number of AI server nodes; vertical resource quota adjustment refers to adjusting the resource allocation quota (such as GPU memory, CPU core count, network bandwidth, etc.) of a single AI server.
[0060] Optionally, in some embodiments, step S104 above includes: Obtain the tiered service quality requirements corresponding to the computing power tasks; The computing power tasks are scheduled and allocated using the target resource allocation strategy to generate an initial computing power task allocation scheme; Based on the tiered service quality requirements, the initial computing power task allocation scheme was adjusted to generate computing power task allocation schemes for each AI server.
[0061] For example, in this embodiment, we designed a hierarchical topology-aware scheduling algorithm for communication-intensive tasks such as distributed training. The algorithm first analyzes the communication patterns of the tasks: Parameter server architecture: Allocate appropriate bandwidth resources to the parameter server and worker nodes; All-Reduce architecture: Selects the best network topology match based on traffic volume; The algorithm constructs a multi-level topology graph of the cluster, from intra-rack topology and cross-rack topology to the overall cluster topology, and then uses graph embedding techniques to encode the topology information into low-dimensional vectors. During scheduling, the algorithm matches the communication patterns of tasks with the cluster topology based on similarity, prioritizing the node combination with the best-matching topology structure.
[0062] For tasks requiring multi-GPU collaboration, the algorithm also considers the connection methods between GPUs (such as NVLink and PCIe), and uses a bipartite graph maximum weight matching algorithm to map the GPU requirements in the task to physical GPUs, maximizing the utilization of high-speed interconnects.
[0063] In this embodiment, the system establishes a four-level service quality assurance system: Platinum-level service (online inference critical path): Absolute resource guarantee, supports millisecond-level elastic scaling, and enjoys the highest preemption priority.
[0064] Gold-tier service (interactive training and development): Resources are highly guaranteed, response time is in the seconds range, and it may be downgraded only in extreme cases.
[0065] Silver-level service (batch training tasks): Elastic resource allocation allows for resource reclamation and rescheduling, ensuring the final completion of the job.
[0066] Bronze-level service (low-priority task): runs using idle resources, can be preempted at any time, and has no service level guarantee.
[0067] When a high-priority task requires resources, the system executes a refined preemption process: Candidate selection: The most suitable preemption candidate task is selected based on a multidimensional evaluation function, taking into account factors such as task running time, checkpoint frequency, resource usage, and priority.
[0068] State preservation: Sending a graceful termination signal to the preempted task triggers application-layer state preservation. The system provides a layered checkpoint mechanism: Lightweight checkpoints: Only save critical states; fast but recovery may not be complete. Full checkpoint: Saves the complete state, takes longer but recovery is reliable; Resource reclamation: After waiting for the maximum tolerance time, forcibly reclaim resources and record the task context.
[0069] Task recovery: When resources are available, intelligently select the timing and location for recovery to minimize interference with the cluster.
[0070] For example, in this embodiment, for distributed training tasks, the system first collects data such as GPU memory utilization, NVLink connection status, and network bandwidth of each server through a global monitoring module; predicts load changes for the next hour based on a multi-scale spatiotemporal attention prediction model; generates an initial scheduling strategy (prioritizing nodes with sufficient GPU resources and tight network connections) through a multi-objective constraint reinforcement learning algorithm; optimizes cross-node communication costs through a hypergraph partitioning algorithm; and finally generates a task allocation scheme to allocate different sub-tasks of the training task to server nodes in adjacent racks, thereby reducing communication latency.
[0071] Through the above technical solution, a full-domain monitoring module collects multi-dimensional resource status data and workload data corresponding to the AI server cluster in real time. The multi-dimensional resource status data indicates whether the multi-dimensional resources of each AI server are currently in an active or dormant state, while the workload data indicates the overall load of the multi-dimensional resources of each AI server. Based on the workload data and the historical load data of each AI server, a multi-scale spatiotemporal attention prediction model is used to dynamically predict the short-term and long-term workloads of each AI server, generating predicted load data for each AI server. Based on the predicted load data, multi-dimensional resource status data, and workload data, a multi-objective constraint reinforcement learning algorithm is used to adaptively adjust the computing power of each AI server cluster, generating an initial scheduling strategy for the current computing power tasks of the AI server cluster. A resource allocation optimization algorithm based on a hypergraph model is used to fine-tune the initial scheduling strategy, generating a target resource allocation strategy for the AI server cluster. The computing power tasks are then scheduled and allocated using the target resource allocation strategy, generating a computing power task allocation scheme for each AI server. This enables adaptive adjustment of the computing power of the AI server cluster. Based on the relationship between the current computing power and load of different AI servers, the computing power task allocation scheme of different AI servers can be dynamically adjusted, making full use of the computing power resources of each AI server and improving the task computing efficiency of the AI server cluster.
[0072] Figure 2 This is a block diagram illustrating an adaptive scheduling device for computing power of an AI server cluster according to an exemplary embodiment. Figure 2 As shown, the device 100 includes: The acquisition module 110 is used to collect multi-dimensional resource status data and workload data corresponding to the AI server cluster in real time. The multi-dimensional resource status data is used to indicate whether the multi-dimensional resources corresponding to each AI server are currently in an enabled or dormant state. The workload data is used to indicate the overall load of the multi-dimensional resources corresponding to each AI server. The first generation module 120 is used to dynamically predict the short-term and long-term workloads of each AI server based on the workload data and the historical load data of each AI server, using a multi-scale spatiotemporal attention prediction model, and generate predicted load data corresponding to each AI server. The second generation module 130 is used to adaptively adjust the computing power of each AI server cluster based on the predicted load data, the multi-dimensional resource status data and the workload data through a multi-objective constraint reinforcement learning algorithm, and generate an initial scheduling strategy for the AI server cluster corresponding to the current computing power task. The execution module 140 is used to perform a resource allocation optimization algorithm based on a hypergraph model to finely adjust the initial scheduling strategy, generate a target resource allocation strategy for the AI server cluster, and schedule and allocate the computing power tasks through the target resource allocation strategy to generate a computing power task allocation scheme for each AI server. In the hypergraph model, a hypernode represents an AI server node, and a hyperedge represents an AI computing task and connects all AI server nodes that meet the resource requirements of the task.
[0073] Optionally, in some embodiments, the first generation module 120 includes: The first generation submodule is used to decompose the historical load data and generate trend terms, periodic terms and residual terms corresponding to the historical load data. Multinomial regression, Fourier series decomposition and long short-term memory network are used for modeling to generate a cluster physical topology and network connection relationship model, wherein the cluster physical topology and network connection relationship model is a graph model. The second generation submodule is used to capture the associations between nodes in the cluster physical topology and network connection relationship model through graph convolutional networks, so as to generate the load association relationship and propagation feature relationship between the nodes. The third generation submodule is used to fuse the temporal features and topology association features in the cluster physical topology and network connection relationship model through a spatiotemporal attention mechanism based on the load correlation relationship and the propagation feature relationship, and generate multi-step predicted load results for the future load data corresponding to each AI server. A determination submodule is used to determine the predicted load data of each AI server based on the multi-step predicted load results using an adaptive selector based on ensemble learning.
[0074] Optionally, in some embodiments, the second generation submodule is used for: Obtain the neighboring nodes between each node in the cluster physical topology and network connection relationship model, as well as the connection relationship between each node and its neighboring nodes; Based on the adjacent nodes and the connection relationships, the load association relationship and the propagation characteristic relationship between each node are determined.
[0075] Optionally, in some embodiments, the execution module 140 is used for; Obtain the hypergraph model, wherein the hypernodes in the hypergraph model represent each AI server node, the hypergraph model includes a multi-dimensional resource attribute vector, the hyperedges represent AI computing tasks, and connect all AI server nodes that can meet the resource requirements of the task. The resource allocation problem is formalized as a constrained hypergraph balancing partitioning problem. The optimization objective is to minimize the cross-node communication cost while satisfying the task resource requirements and maintaining load balancing among nodes, so as to generate a hypergraph partitioning algorithm to be solved. Based on the initial scheduling strategy, an iteratively improved hierarchical approach is used to solve the hypergraph partitioning algorithm to generate the optimal partitioning method corresponding to the hypergraph model. Based on the optimal partitioning method and the corresponding set of AI server nodes, the target resource allocation strategy for the AI server cluster is generated.
[0076] Optionally, in some embodiments, the device 100 further includes a generation module, which is used for: Based on the target resource allocation strategy, the multi-scale spatiotemporal attention prediction model and the reinforcement learning algorithm for multi-objective constraints are optimized to generate a target multi-scale spatiotemporal attention prediction model and a target multi-objective constraint reinforcement learning algorithm.
[0077] Optionally, in some embodiments, the execution module 140 is configured to: Obtain real-time data metrics for each AI server; Based on the target resource allocation strategy and the real-time data indicators, the computing power of each AI server is adjusted by horizontally expanding the number of replicas or vertically adjusting the resource limit. Based on the computing power task and the horizontal replica number expansion or vertical resource limit adjustment, determine the AI server nodes that need to be added or removed in the AI server cluster, as well as the computing power tasks that need to be split. Based on the computing power tasks that need to be split and the AI server nodes that need to be added or removed, a computing power task allocation scheme is generated for each AI server.
[0078] Optionally, in some embodiments, the execution module 140 is configured to: Obtain the graded service quality requirements corresponding to the computing power task; The computing power tasks are scheduled and allocated using the target resource allocation strategy to generate an initial computing power task allocation scheme; Based on the tiered service quality requirements, the initial computing power task allocation scheme is adjusted to generate the computing power task allocation scheme corresponding to each AI server.
[0079] Through the above technical solution, multi-dimensional resource status data and workload data corresponding to the AI server cluster are collected in real time. The multi-dimensional resource status data indicates whether the multi-dimensional resources of each AI server are currently in an active or dormant state, and the workload data indicates the overall load of the multi-dimensional resources of each AI server. Based on the workload data and the historical load data of each AI server, a multi-scale spatiotemporal attention prediction model is used to dynamically predict the short-term and long-term workloads of each AI server, generating predicted load data for each AI server. Based on the predicted load data, multi-dimensional resource status data, and workload data, a multi-objective constraint reinforcement learning algorithm is used to adaptively adjust the computing power of each AI server cluster, generating an initial scheduling strategy for the current computing power tasks of the AI server cluster. A resource allocation optimization algorithm based on a hypergraph model is used to fine-tune the initial scheduling strategy, generating a target resource allocation strategy for the AI server cluster. The computing power tasks are then scheduled and allocated according to the target resource allocation strategy, generating a computing power task allocation scheme for each AI server. This enables adaptive adjustment of the computing power of the AI server cluster. Based on the relationship between the current computing power and load of different AI servers, the computing power task allocation scheme of different AI servers can be dynamically adjusted, making full use of the computing power resources of each AI server and improving the task computing efficiency of the AI server cluster.
[0080] Figure 3 This is a block diagram illustrating an electronic device 300 according to an exemplary embodiment. Figure 3 As shown, the electronic device 300 may include a processor 301 and a memory 302. The electronic device 300 may also include one or more of a multimedia component 303, an input / output (I / O) interface 304, and a communication component 305.
[0081] The processor 301 controls the overall operation of the electronic device 300 to complete all or part of the steps in the aforementioned AI server cluster computing power adaptive scheduling method. The memory 302 stores various types of data to support the operation of the electronic device 300. This data may include, for example, instructions for any application or method operating on the electronic device 300, and application-related data such as contact data, sent and received messages, images, audio, video, etc. The memory 302 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. Multimedia component 303 may include a screen and an audio component. The screen may be, for example, a touchscreen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in memory 302 or transmitted via communication component 305. The audio component also includes at least one speaker for outputting audio signals. I / O interface 304 provides an interface between processor 301 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual or physical buttons. Communication component 305 is used for wired or wireless communication between the electronic device 300 and other devices. Wireless communication may include Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, or 4G, or a combination of these. Therefore, the corresponding communication component 305 may include a Wi-Fi module, a Bluetooth module, or an NFC module.
[0082] In an exemplary embodiment, the electronic device 300 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to execute the above-described adaptive scheduling method for the computing power of the AI server cluster.
[0083] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided. When executed by a processor, these program instructions implement the steps of the above-described adaptive scheduling method for the computing power of an AI server cluster. For example, the computer-readable storage medium may be the memory 302 including the program instructions, which may be executed by the processor 301 of the electronic device 300 to complete the above-described adaptive scheduling method for the computing power of an AI server cluster.
[0084] In another exemplary embodiment, a computer program product is also provided, which includes a computer program executable by a processor, which, when executed by the processor, implements the steps of the above-described AI server cluster computing power adaptive scheduling method.
[0085] In another exemplary embodiment, a computer program product is also provided, which includes a computer program executable by a processor, which, when executed by the processor, implements the steps of the above-described AI server cluster computing power adaptive scheduling method.
[0086] The preferred embodiments of this disclosure have been described in detail above with reference to the accompanying drawings. However, this disclosure is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this disclosure, various simple modifications can be made to the technical solutions of this disclosure, and these simple modifications all fall within the protection scope of this disclosure.
[0087] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction.
[0088] Furthermore, various different embodiments of this disclosure can be combined in any way, as long as they do not violate the spirit of this disclosure, they should also be regarded as the content disclosed in this disclosure.
Claims
1. A method for adaptive scheduling of computing power in an AI server cluster, characterized in that, The method includes: Real-time collection of multi-dimensional resource status data and workload data corresponding to the AI server cluster. The multi-dimensional resource status data is used to indicate whether the multi-dimensional resources corresponding to each AI server are currently in an enabled or dormant state. The workload data is used to indicate the overall load of the multi-dimensional resources corresponding to each AI server. Based on the workload data and the historical load data of each AI server, the short-term and long-term workloads of each AI server are dynamically predicted using a multi-scale spatiotemporal attention prediction model, generating predicted load data for each AI server. Based on the predicted load data, the multi-dimensional resource status data, and the workload data, the computing power of each AI server cluster is adaptively adjusted through a reinforcement learning algorithm with multi-objective constraints to generate an initial scheduling strategy for the current computing power task of the AI server cluster. The multi-objective constraints include at least one of the following: load balancing constraints, communication cost constraints, and task resource requirement constraints. The resource allocation optimization algorithm based on the hypergraph model finely adjusts the initial scheduling strategy to generate the target resource allocation strategy for the AI server cluster. The computing power tasks are then scheduled and allocated using the target resource allocation strategy to generate a computing power task allocation scheme for each AI server. In the hypergraph model, hypernodes represent AI server nodes, and hyperedges represent AI computing tasks and connect all AI server nodes that meet the resource requirements of the task.
2. The adaptive scheduling method for computing power of an AI server cluster according to claim 1, characterized in that, The step involves dynamically predicting the short-term and long-term workloads of each AI server based on the workload data and historical load data of each AI server, using a multi-scale spatiotemporal attention prediction model. This generates predicted load data for each AI server, including: The historical load data is decomposed to generate trend terms, periodic terms and residual terms corresponding to the historical load data. Multinomial regression, Fourier series decomposition and long short-term memory network are used to model the cluster physical topology and network connection relationship model, wherein the cluster physical topology and network connection relationship model is a graph model. The graph convolutional network is used to capture the relationships between nodes in the cluster physical topology and network connection model, so as to generate the load relationship and propagation characteristic relationship between the nodes. Based on the load correlation relationship and the propagation feature relationship, the temporal features and topology correlation features in the cluster physical topology and network connection relationship model are fused through a spatiotemporal attention mechanism to generate multi-step predicted load results for the future load data of each AI server. Based on the multi-step predicted load results, the predicted load data for each AI server is determined by an adaptive selector based on ensemble learning.
3. The adaptive scheduling method for computing power of an AI server cluster according to claim 2, characterized in that, The method of capturing the relationships between nodes in the cluster physical topology and network connection model through graph convolutional networks to generate load relationships and propagation characteristic relationships between nodes includes: Obtain the neighboring nodes between each node in the cluster physical topology and network connection relationship model, as well as the connection relationship between each node and its neighboring nodes; Based on the adjacent nodes and the connection relationships, the load association relationship and the propagation characteristic relationship between each node are determined.
4. The adaptive scheduling method for computing power of an AI server cluster according to claim 1, characterized in that, The resource allocation optimization algorithm based on the hypergraph model finely adjusts the initial scheduling strategy to generate the target resource allocation strategy for the AI server cluster, including: Obtain the hypergraph model, wherein the hypernodes in the hypergraph model represent each AI server node, the hypergraph model includes a multi-dimensional resource attribute vector, the hyperedges represent AI computing tasks, and connect all AI server nodes that can meet the resource requirements of the task. The resource allocation problem is formalized as a constrained hypergraph balancing partitioning problem. The optimization objective is to minimize the cross-node communication cost while satisfying the task resource requirements and maintaining load balancing among nodes, so as to generate a hypergraph partitioning algorithm to be solved. Based on the initial scheduling strategy, an iteratively improved hierarchical approach is used to solve the hypergraph partitioning algorithm to generate the optimal partitioning method corresponding to the hypergraph model. Based on the optimal partitioning method and the corresponding set of AI server nodes, the target resource allocation strategy for the AI server cluster is generated.
5. The adaptive scheduling method for computing power of an AI server cluster according to any one of claims 1-4, characterized in that, The method further includes: Based on the target resource allocation strategy, the multi-scale spatiotemporal attention prediction model and the reinforcement learning algorithm for multi-objective constraints are optimized to generate a target multi-scale spatiotemporal attention prediction model and a target multi-objective constraint reinforcement learning algorithm.
6. The adaptive scheduling method for computing power of an AI server cluster according to claim 5, characterized in that, The step of scheduling and allocating computing power tasks according to the target resource allocation strategy to generate computing power task allocation schemes for each AI server includes: Obtain real-time data metrics for each AI server; Based on the target resource allocation strategy and the real-time data indicators, the computing power of each AI server is adjusted by horizontally expanding the number of replicas or vertically adjusting the resource limit. Based on the computing power task and the horizontal replica number expansion or vertical resource limit adjustment, determine the AI server nodes that need to be added or removed in the AI server cluster, as well as the computing power tasks that need to be split. Based on the computing power tasks that need to be split and the AI server nodes that need to be added or removed, a computing power task allocation scheme is generated for each AI server.
7. The adaptive scheduling method for computing power of an AI server cluster according to claim 5, characterized in that, The step of scheduling and allocating computing power tasks according to the target resource allocation strategy to generate computing power task allocation schemes for each AI server includes: Obtain the graded service quality requirements corresponding to the computing power task; The computing power tasks are scheduled and allocated using the target resource allocation strategy to generate an initial computing power task allocation scheme; Based on the tiered service quality requirements, the initial computing power task allocation scheme is adjusted to generate the computing power task allocation scheme corresponding to each AI server.
8. A computing power adaptive scheduling device for an AI server cluster, characterized in that, The device includes: The acquisition module is used to collect multi-dimensional resource status data and workload data corresponding to the AI server cluster in real time. The multi-dimensional resource status data is used to indicate whether the multi-dimensional resources corresponding to each AI server are currently in an enabled or dormant state. The workload data is used to indicate the overall load of the multi-dimensional resources corresponding to each AI server. The first generation module is used to dynamically predict the short-term and long-term workloads of each AI server based on the workload data and the historical load data of each AI server, using a multi-scale spatiotemporal attention prediction model, and generate predicted load data for each AI server respectively. The second generation module is used to adaptively adjust the computing power of each AI server cluster based on the predicted load data, the multi-dimensional resource status data, and the workload data through a reinforcement learning algorithm with multi-objective constraints, and generate an initial scheduling strategy for the AI server cluster corresponding to the current computing power task. The multi-objective constraints include at least one of the following: load balancing constraints, communication cost constraints, and task resource requirement constraints. The execution module is used to refine the initial scheduling strategy based on the resource allocation optimization algorithm of the hypergraph model, generate the target resource allocation strategy of the AI server cluster, and schedule and allocate the computing power tasks through the target resource allocation strategy to generate the computing power task allocation scheme corresponding to each AI server. In the hypergraph model, the hypernode represents the AI server node, and the hyperedge represents the AI computing task and connects all AI server nodes that meet the resource requirements of the task.
9. An electronic device, characterized in that, include: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the method according to any one of claims 1-7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method described in any one of claims 1-7.