AI computing power platform based on elastic telescoping mechanism

Abstract

The invention relates to the technical field of artificial intelligence, and discloses an AI computing power platform based on an elastic scaling mechanism, comprising: a task queue module maintaining a dynamic task list composed of user tasks submitted by a plurality of user sides; the historical operation log database collects historical task log data of user task multi-dimensional resources; the task resource prediction module predicts a multi-dimensional resource demand according to the historical operation log data; the AI computing power scheduling module deploys the tasks to server nodes capable of meeting resource requirements by adopting a preset AI computing power resource scaling scheduling algorithm according to the task resource prediction result; and the state synchronization feedback module updates the task execution state and the cluster resource state and feeds back the state change. According to the method, the problem of balancing the resource utilization rate and the task performance is solved, the intelligence and the efficiency of AI task scheduling in a large-scale cluster environment are improved, the resource management requirements of real-time and high-concurrency tasks are met, and a better resource scheduling solution is provided for AI application.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to an AI computing power platform based on an "elastic scaling" mechanism. Background Technology

[0002] In cloud computing and artificial intelligence task scheduling scenarios, traditional resource allocation strategies have significant drawbacks: On the one hand, the static resource request mode at task submission leads to resource waste, such as excessive requests for unused computing and storage resources, or performance bottlenecks, such as insufficient allocation of critical resources. Especially in AI tasks, such as model training and inference, different tasks have significantly different sensitivities to resources such as CPU, GPU, and memory, making static allocation difficult to adapt to dynamic needs. On the other hand, the lack of intelligent modeling of the relationship between resource scaling and task performance makes it impossible to guarantee performance within acceptable thresholds when resources are reduced, resulting in low scheduling efficiency. In existing technologies, although task queue management, resource allocation, and state update processes cover the entire scheduling lifecycle, they do not deeply integrate AI-driven adaptive resource adjustment, nor do they optimize deployment strategies for the heterogeneity of cluster resources.

[0003] Therefore, there is an urgent need for a scheduling method that integrates task performance prediction, resource importance analysis, dynamic scaling adjustment, and cluster resource status awareness. Summary of the Invention

[0004] This invention provides an AI computing platform based on an "elastic scaling" mechanism, which solves the problem of balancing resource utilization and task performance, improves the intelligence and efficiency of AI task scheduling in large-scale cluster environments, meets the resource management needs of real-time and high-concurrency tasks, and provides a better resource scheduling solution for cloud computing, big data and AI applications.

[0005] This invention provides an AI computing power platform based on an "elastic scaling" mechanism, including a task queue module, an AI computing power scheduling module, a task resource prediction module, a historical operation log database, a status synchronization feedback module, multiple user terminals, and multiple server nodes. The AI ​​computing power scheduling module is connected to the task resource prediction module, the task queue module, the status synchronization feedback module, and the multiple server nodes. The task queue module is also connected to the multiple user terminals, and the task resource prediction module is also connected to the historical operation log database.

[0006] The task queue module is used to maintain a dynamic task list consisting of user tasks submitted by the multiple user terminals, including original tasks and updated tasks;

[0007] The historical operation log database is used to collect historical task log data of user tasks in multiple dimensions, including resource usage, execution time, and task type, as the data foundation for the task resource prediction module.

[0008] The task resource prediction module is used to predict multi-dimensional resource requirements based on the historical operation log data and output the results to the AI ​​computing power scheduling module.

[0009] The AI ​​computing power scheduling module is used to deploy tasks to server nodes that can meet resource requirements based on task resource prediction results and a preset AI computing power resource scaling scheduling algorithm, thereby realizing dynamic resource scheduling of cluster job resource allocation.

[0010] The status synchronization feedback module is used to update the task execution status and cluster resource status, and to report the status changes to the task queue module and the AI ​​computing power scheduling module.

[0011] Furthermore, the task resource prediction module extracts AI computing cluster log data from the historical operation log database, selects m features and n samples of multi-dimensional resource usage from the AI ​​computing cluster log data, and constructs a regression model of task resource requirements under multi-dimensional resources. The feature values ​​of the regression model data include the number of CPU cores, GPU percentage utilization, memory capacity requirements, network traffic, and video memory percentage utilization. The target value of the model data is y. i Let be the execution time of the i-th sample; where the AI ​​computing power cluster log data includes resource data and task type data in multiple dimensions during task execution, and the multiple dimensions of resource data specifically include GPU request amount, CPU request amount, memory request amount, network bandwidth, and IO usage;

[0012] The XGBoost machine learning method is used for modeling regression. The output expression of the task resource demand prediction model based on XGBoost is:

[0013]

[0014] in, t represents the predicted value after t iterations, i.e., the task execution time predicted by the model based on resource conditions; k is the number of decision trees obtained through iteration; and f is the output function of each decision tree.

[0015] The objective function of the task resource requirement prediction model based on XGBoost is:

[0016]

[0017] The task resource requirement model based on XGBoost improves the accuracy of the decision tree model by iterating the residuals of the fitted model t times; where λ and γ are hyperparameters, T represents the number of leaf nodes, g is the first-order partial derivative, h is the second-order partial derivative, and l is the loss function that measures the model fit.

[0018] The modeling process of the task resource demand prediction model establishes the model structure based on minimizing the objective function value. It then selects the feature value splitting point of the leaf node by maximizing the branch gain, sequentially constructing new branches of the tree model for each feature in the model. Finally, it completes the modeling of the task resource demand XGBoost model. The branch gain is the difference between the objective function value before and after the feature value splitting of the leaf node, expressed as:

[0019]

[0020] Each time, the feature with the largest branch gain is selected as the splitting feature of the leaf node. Resource features are selected in turn to complete the construction of the task resource demand prediction model, so that the overall objective function of the model is minimized. Finally, the task resource demand prediction model is established to complete the data fitting prediction.

[0021] Furthermore, the AI ​​computing power scheduling module defines the problem of multi-dimensional resource scheduling of AI computing power as:

[0022] Suppose there exists an AI computing cluster with a heterogeneous server list S = {S1, S2, ..., S...} with M different resource quotas. M The R-dimensional resource capacity corresponding to the i-th server is represented as follows: The resource request amount corresponding to the j-th user task is The formal definition of the resource allocation and scheduling problem is:

[0023]

[0024] Wherein, variable x i,j This indicates that the j-th user task is assigned to the i-th server. During the scheduling process, tasks can be distributed across multiple servers, so x i,j It can be a fraction; p j It represents the task execution benefit of task j in the current scheduling round, and the scheduling goal is to maximize the task execution benefit in the current scheduling round.

[0025] Furthermore, in the AI ​​computing power scheduling module, a preset AI computing power resource scaling scheduling algorithm is set according to the problem definition as follows:

[0026] S1. Extract the tasks that need to be scheduled from the task pool and update the task queue for the current round; wherein, the task queue includes online tasks and tasks to be processed;

[0027] S2. If there are no tasks in the task queue of the current round, the scheduling of this round will end directly.

[0028] S2. If there are tasks in the task queue of the current round, sort the task queue of the current round according to the preset scheduling strategy.

[0029] S3. For each task, an adaptive resource scaling algorithm is used to perform adaptive resource scaling based on the task type, and the resource request quantity is updated and adjusted.

[0030] S4. Sort the servers according to the amount of available resources from most to least;

[0031] S5. Check if there are servers that can meet the resource requirements of the task. If so, extract the server set, deploy the task to the server set, and update the server status. If not, put the task into the pending queue and wait for it to be scheduled again in the next scheduling round.

[0032] S6. Mark the status of the task to update the task queue status, so as to facilitate the next round of scheduling.

[0033] Furthermore, in step S3, the preset adaptive resource scaling algorithm includes:

[0034] S301, According to task type C j Load the task resource requirement prediction model;

[0035] S302. Calculate the amount of resources w requested by the current user. j Expected performance of the task t j ;

[0036] S303. Calculate the importance of each resource type to task performance based on the task resource requirement prediction model, and generate an importance list;

[0037] S304. When the adaptive adjustment rate α < 1, for each resource r, if r is not on the importance list, its application quantity is reduced proportionally to α.

[0038] S305. After adjustment, calculate the new resource W. j Performance T j ;

[0039] S306, Calculation error = |T j -t j If error < θ, then return the adjusted resource W. j Otherwise, increase α, i.e., α = α + Δα, and try to reduce more non-critical resources again.

[0040] S307. When the adaptive adjustment rate α > 1, if the error requirement still cannot be met, then abandon scaling and return to the original resource request amount w. j .

[0041] Furthermore, it also includes a resource monitoring module, which is connected to the AI ​​computing power scheduling module. The resource monitoring module is used to collect multi-dimensional resource data from multiple server nodes in real time to generate a resource topology map and to perform anomaly detection and analysis on the resource data. The multi-dimensional resource data includes CPU utilization, GPU memory usage, memory usage, network bandwidth, and storage I / O.

[0042] Furthermore, the resource monitoring module performs anomaly detection and analysis on resource data, including anomaly detection based on preset rules and anomaly detection based on machine learning.

[0043] Anomaly detection based on preset rules includes:

[0044] If CPU utilization is set to be greater than a first set value and remains so for a first set duration, it is marked as node overload; if GPU memory usage is set to be less than a second set value and the task queue is backlogged, it is marked as resource waste.

[0045] If CPU utilization is set to be greater than the third setting and network bandwidth is set to be less than the fourth setting, then a computing-network bottleneck is identified, and task migration is triggered.

[0046] The sliding window mean of the metric is calculated and compared with the historical average. If the difference exceeds the first preset threshold, it is marked as abnormal.

[0047] Machine learning-based anomaly detection includes:

[0048] An autoencoder is used to train a model to reconstruct normal resource data. Samples with reconstruction errors exceeding a second preset threshold are judged as abnormal.

[0049] Furthermore, it also includes a task priority management module, which is connected to the status synchronization and feedback module. The task priority management module is used to assign priority tags to tasks and prioritize high-priority tasks during scheduling.

[0050] The methods for determining task priorities include: directly defining priorities by the tags, types, and user levels provided when the task is submitted; dynamically adjusting priorities based on resource utilization, execution progress, and dependencies during task execution; or manually adjusting task priorities by administrators or users.

[0051] The beneficial effects of this invention are as follows:

[0052] This invention achieves adaptive scaling and adjustment of resources across multiple dimensions by constructing a task performance prediction model and a resource importance analysis mechanism, significantly improving cluster resource utilization and task scheduling efficiency. Based on task type matching model prediction and resource importance ranking, it can dynamically reduce the amount of non-critical resource requests while ensuring task performance, effectively reducing resource waste and making cluster resource allocation more aligned with actual task needs, thus improving overall resource utilization. Through task queue sorting, server resource status awareness, and optimized deployment strategies, it can prioritize matching high-priority tasks with high-availability resource nodes, reducing task waiting time and resource fragmentation, enhancing the system's ability to handle real-time, high-concurrency AI tasks, and ultimately achieving intelligent, efficient, and refined cluster resource scheduling. This provides a superior resource scheduling solution for task management in cloud computing, big data, and artificial intelligence scenarios, significantly reducing operating costs. Attached Figure Description

[0053] Figure 1 This is a schematic diagram of the AI ​​computing platform based on the "elastic scaling" mechanism of the present invention.

[0054] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0055] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0056] like Figure 1 As shown, this invention provides an AI computing power platform based on an "elastic scaling" mechanism, including a task queue module, an AI computing power scheduling module, a task resource prediction module, a historical operation log database, a status synchronization feedback module, a resource monitoring module, a task priority management module, multiple user terminals, and multiple server nodes. The AI ​​computing power scheduling module is connected to the task resource prediction module, the task queue module, the resource monitoring module, the status synchronization feedback module, and the multiple server nodes. The task queue module is also connected to the multiple user terminals. The task resource prediction module is also connected to the historical operation log database. The task priority management module is connected to the status synchronization and feedback module.

[0057] First, the AI ​​computing power scheduling module updates the dynamic task queue based on the tasks dynamically submitted by the user, as well as specific task attributes such as task type and multi-dimensional resource requirements. Next, in the current scheduling round, on the task side, the AI ​​computing power scheduling module, combined with a task resource requirement prediction model, predicts the expected performance of the user's current task requesting resources in the task queue. Based on the expected performance, it adaptively scales and adjusts the multi-dimensional resource requirements, returning the updated task resource requirements to the AI ​​computing power scheduling module. Then, on the cluster resource side, the AI ​​computing power scheduling module, combining the task requirement scaling results and the cluster's multi-dimensional resource usage status, deploys the task to server nodes that can meet the resource requirements, dynamically optimizing the resource allocation of cluster jobs. Finally, it updates the execution status at the task level and the resource status at the cluster level, preparing for the next scheduling round. By considering the resource requirement characteristics of different task types and fully utilizing the cluster's multi-dimensional resources, the mismatch between multiple dimensions of resource requests can be resolved, improving the deployment efficiency of tasks at the cluster level and the utilization rate of multi-dimensional resources.

[0058] (1) Task Queue Module

[0059] The task queue module is used to maintain a dynamic task list consisting of user tasks submitted by the multiple user terminals, including original tasks and update tasks.

[0060] (2) Historical operation log database

[0061] The historical operation log database is used to collect historical task log data on multi-dimensional resource usage, execution time, and task type of user tasks, serving as the data foundation for the task resource prediction module.

[0062] (3) Task resource prediction module

[0063] The task resource prediction module is used to predict multi-dimensional resource requirements based on the historical operation log data and output the results to the AI ​​computing power scheduling module.

[0064] For AI computing power scheduling, task resource requirements specifically include GPU allocation, CPU allocation, memory allocation, network bandwidth, I / O usage, etc. AI tasks are categorized into three types: GPU-intensive, CPU-intensive, and memory-intensive. The resource requirements involved in the scheduling process focus on these three dimensions: GPU, CPU, and memory. GPU-intensive neural network machine translation tasks require significant GPU resources. Conversely, CPU-intensive recommendation tasks waste 56% of their overall execution time waiting for training data input, sometimes exceeding 80%. Memory-intensive image classification tasks incur significant memory access overhead during data reading, with 10%–70% of the training time spent on data retrieval. In other words, the efficiency of GPU resource utilization for these tasks is affected by other resource dimensions (such as CPU and memory). Therefore, considering the multi-dimensional resource impact, a task resource requirement prediction model is established, and AI computing power scheduling strategies are designed based on the multi-dimensional resource characteristics of different task types.

[0065] The task resource prediction module extracts AI computing cluster log data from the historical operation log database, selects m features and n samples of multi-dimensional resource usage from the AI ​​computing cluster log data, and constructs a regression model of task resource requirements under multi-dimensional resources. The feature values ​​of the regression model data include the number of CPU cores, GPU percentage utilization, memory capacity requirements, network traffic, and video memory percentage utilization. The target value of the model data is y. i Let be the execution time of the i-th sample; where the AI ​​computing power cluster log data includes resource data and task type data in multiple dimensions during task execution, and the multiple dimensions of resource data specifically include GPU request amount, CPU request amount, memory request amount, network bandwidth, and IO usage;

[0066] The modeling regression process is based on the Extreme Gradient Boosting (XGBoost) machine learning method to analyze the multi-dimensional resource requirements of different types of tasks, i.e., the impact of multi-dimensional resources on task execution time. XGBoost trains a set of decision trees based on the boosting method and iteratively fits the residuals of the decision trees using gradient boosting. The output of the XGBoost model is a weighted sum of the predictions from all residual iterative decision trees. The output expression of the task resource requirement prediction model based on XGBoost is:

[0067]

[0068] in, t represents the predicted value after t iterations, i.e., the task execution time predicted by the model based on resource conditions; k is the number of decision trees obtained through iteration; and f is the output function of each decision tree.

[0069] The task resource requirement model based on XGBoost improves the accuracy of the decision tree model by iterating the residuals of the model t times, and obtains the iterative objective function expression through second-order Taylor series.

[0070]

[0071] Where Ω represents the regularization factor introduced to reduce error loss and is used to measure the model's complexity, g is the first-order partial derivative, and h is the second-order partial derivative, as shown in the following formula. The extreme points of this function are obtained as follows:

[0072]

[0073] The extreme value of the objective function of the XGBoost-based task resource requirement prediction model is:

[0074]

[0075] The task resource requirement model based on XGBoost improves the accuracy of the decision tree model by iterating the residuals of the model t times; where λ and γ are hyperparameters, T represents the number of leaf nodes, and l is the loss function that measures the model fit.

[0076] The modeling process of the task resource demand prediction model establishes the model structure based on minimizing the objective function value. It then selects the feature value splitting point of the leaf node by maximizing the branch gain, sequentially constructing new branches of the tree model for each feature in the model. Finally, it completes the modeling of the task resource demand XGBoost model. The branch gain is the difference between the objective function value before and after the feature value splitting of the leaf node, expressed as:

[0077]

[0078] Each time, the feature with the largest branch gain is selected as the splitting feature of the leaf node. Resource features are selected in turn to complete the construction of the task resource demand prediction model, so that the overall objective function of the model is minimized. Finally, the task resource demand prediction model is established to complete the data fitting prediction.

[0079] (4) AI computing power scheduling module

[0080] The AI ​​computing power scheduling module is used to deploy tasks to server nodes that can meet resource requirements based on task resource prediction results and a preset AI computing power resource scaling scheduling algorithm, thereby realizing dynamic resource scheduling of cluster operations.

[0081] The AI ​​computing power scheduling module defines the problem of multi-dimensional resource scheduling of AI computing power as follows:

[0082] Suppose there exists an AI computing cluster with a heterogeneous server list S = {S1, S2, ..., S...} with M different resource quotas. M The R-dimensional resource capacity corresponding to the i-th server is represented as follows: The resource request amount corresponding to the j-th user task is The formal definition of the resource allocation and scheduling problem is:

[0083]

[0084] Wherein, variable x i,j This indicates that the j-th user task is assigned to the i-th server. During the scheduling process, tasks can be distributed across multiple servers, so x i,k It can be a fraction; p j This represents the task execution benefit of task j in the current scheduling round. The scheduling objective is to maximize the task execution benefit in the current scheduling round. Simultaneously, the task's resource requirements should be less than the available resources of the service nodes, ensuring that the resources allocated to the task do not exceed the available resource quotas across multiple dimensions of the server. Furthermore, it must be guaranteed that the user task can be executed within the current set of server nodes.

[0085] The AI ​​computing power adaptive resource scaling and scheduling algorithm combines a task resource demand prediction model to analyze the impact of multi-dimensional resource allocation on task performance, adaptively scales and adjusts the multi-dimensional resource demands of tasks in the current dynamic queue, and finally deploys the tasks to a suitable cluster location.

[0086] The AI ​​computing power resource scaling and scheduling algorithm first performs resource scaling adjustments on tasks in the scheduling round, then selects and updates the cluster nodes for deployment. First, it filters the list of schedulable task queues for the current scheduling round; second, it adaptively scales and adjusts the resource requirements of tasks based on task type and corresponding resource requirement prediction models; finally, it deploys the tasks on the cluster node servers and updates the cluster status.

[0087] Based on the problem definition, the preset AI computing power resource scaling and scheduling algorithm is set as follows:

[0088] S1. Extract the tasks that need to be scheduled from the task pool and update the task queue for the current round; wherein, the task queue includes online tasks and tasks to be processed;

[0089] S2. If there are no tasks in the task queue of the current round, the scheduling of this round will end directly.

[0090] S2. If there are tasks in the task queue of the current round, sort the task queue of the current round according to the preset scheduling strategy.

[0091] S3. For each task, an adaptive resource scaling algorithm is used to perform adaptive resource scaling based on the task type, and the resource request quantity is updated and adjusted.

[0092] The preset adaptive resource scaling algorithms include:

[0093] S301, According to task type C j Load the task resource requirement prediction model;

[0094] S302. Calculate the amount of resources w requested by the current user. j Expected performance of the task t j ;

[0095] S303. Calculate the importance of each resource type to task performance based on the task resource requirement prediction model, and generate an importance list;

[0096] S304. When the adaptive adjustment rate α < 1, for each resource r, if r is not on the importance list, its application quantity is reduced proportionally to α.

[0097] S305. After adjustment, calculate the new resource W. j Performance T j ;

[0098] S306, Calculation error = |T j -t j If error < θ, then return the adjusted resource W. j Otherwise, increase α, i.e., α = α + Δα, and try to reduce more non-critical resources again.

[0099] S307. When the adaptive adjustment rate α > 1, if the error requirement still cannot be met, then abandon scaling and return to the original resource request amount w. j .

[0100] The adaptive resource scaling algorithm analyzes the feature importance of a task resource demand prediction model to identify resource dimensions that have minimal impact on task performance and are subject to over-allocation. Then, without affecting task performance, it adaptively scales and adjusts the multi-dimensional resources initially requested by the user task, effectively resolving resource over-allocation and mismatch issues between different resource dimensions.

[0101] S4. Sort the servers according to the amount of available resources from most to least;

[0102] S5. Check if there are servers that can meet the resource requirements of the task. If so, extract the server set, deploy the task to the server set, and update the server status. If not, put the task into the pending queue and wait for it to be scheduled again in the next scheduling round.

[0103] S6. Mark the status of the task to update the task queue status, so as to facilitate the next round of scheduling.

[0104] By constructing task characteristics through a task resource requirement model, the impact of multi-dimensional resources on task performance is analyzed, guiding the adaptive dynamic scaling of resource allocation in the GPU cluster during scheduling. This method of dynamically adjusting resource scheduling can optimize resource over-request during user task applications. Simultaneously, this method also optimizes the inefficient utilization of certain resource dimensions in AI tasks due to mismatches between different resource dimensions, ultimately improving the efficiency of actual AI computing power deployment and cluster resource utilization.

[0105] (5) Status synchronization feedback module

[0106] The status synchronization feedback module is used to update the task execution status (such as running, completed, failure reason) and cluster resource status (allocated, remaining resources, node health) to ensure the consistency of the system's global status. Simultaneously, it feeds back status changes to the task queue module and AI computing power scheduling module (such as task timeout triggering resource reallocation, node failure triggering task migration), achieving closed-loop scheduling.

[0107] (6) Resource monitoring module

[0108] The resource monitoring module is used to collect multi-dimensional resource status of server nodes in real time (CPU utilization, GPU memory usage, memory usage, network bandwidth, storage I / O, etc.) and generate resource topology maps (such as node-level and rack-level resource distribution). It also performs anomaly detection and analysis on resource data to provide a real-time and reliable resource view for scheduling and deployment.

[0109] Perform anomaly detection and analysis on resource data, including anomaly detection based on preset rules and anomaly detection based on machine learning;

[0110] a. Anomaly detection based on preset rules includes:

[0111] Single metric threshold: If CPU utilization is greater than the first set value (95%) and continues for the first set duration (5 minutes), it is marked as node overload; if GPU memory usage is less than the second set value (10%) and the task queue is backlogged, it is marked as resource waste.

[0112] Multi-indicator correlation: If CPU utilization is greater than the third setting (90%) and network bandwidth is less than the fourth setting (100Mbps), it is determined to be a computing-network bottleneck, triggering task migration;

[0113] Comparative analysis: Calculate the sliding window mean of the indicator (such as the average CPU value over the past hour) and compare it with the same period in history (such as the same period last week). If the difference exceeds the first preset threshold (30%), it is marked as abnormal (such as a sudden increase in the overall load of the cluster).

[0114] b. Anomaly detection based on machine learning includes:

[0115] An autoencoder is used to train the model to reconstruct normal resource data. Samples with reconstruction errors exceeding a second preset threshold (e.g., MSE > 0.5) are judged as abnormal.

[0116] (7) Task Priority Management Module

[0117] The task priority management module is used to assign priority tags to tasks, prioritize the processing of high-priority tasks during scheduling, and support preemptive scheduling (such as releasing resources from low-priority tasks to high-priority tasks) to improve the response speed of critical tasks and avoid resource starvation.

[0118] The methods for determining task priorities include: directly defining priorities by the tags, types, and user levels provided when the task is submitted; dynamically adjusting priorities based on resource utilization, execution progress, and dependencies during task execution; or manually adjusting task priorities by administrators or users.

[0119] a. Defining priority directly through tags, types, and user levels provided when submitting tasks is suitable for scenarios with clearly defined business rules.

[0120] In terms of task type and business level, tasks with high real-time requirements (such as real-time monitoring and online transactions) have higher priority than batch processing tasks (such as log analysis and offline training). Core business tasks (such as payment systems and user authentication) have higher priority than peripheral business tasks (such as report generation and data backup).

[0121] In user tiers and resource payment models, high-paying users and VIP users have higher task priority than ordinary users. Preemptive resource requests (such as auction instances) or urgent resource applications (such as time-limited tasks) have higher priority.

[0122] In terms of task deadlines and timelines, tasks with closer deadlines (such as report generation with a countdown of 1 hour) have higher priority than tasks with longer deadlines. For periodic tasks, those where missing the execution window would have serious consequences (such as daily financial reconciliation at midnight) are marked as high priority.

[0123] b. Dynamically adjust priority based on resource utilization, execution progress, and dependencies during task execution to adapt to load changes.

[0124] In terms of resource urgency, tasks that wait too long for resources (e.g., queues exceeding 30 minutes) are automatically prioritized to avoid starvation. Tasks with urgent needs for scarce resources (such as GPUs or high-IO disks) are scheduled first.

[0125] In execution status and dependencies, critical path tasks that block other tasks (such as upstream tasks in a data pipeline) have higher priority than independent tasks. Tasks that fail multiple times (e.g., fail ≥3 times) have their priority increased, triggering emergency handling before manual intervention.

[0126] In real-time performance feedback, if a running task exhibits abnormal resource utilization (e.g., CPU utilization consistently >90% and execution speed is slow), it may require more resources, and its priority will be temporarily increased to obtain quotas. When the response latency of interactive tasks (such as real-time user queries) exceeds a threshold, their priority will be dynamically increased for a faster response.

[0127] c. Task priorities can be manually adjusted by administrators or users. A task priority editing function is provided, allowing administrators to mark specific tasks as urgent. Users can request expedited processing when submitting tasks (requiring an approval process); upon approval, the priority is automatically increased. When a high-priority task enters the queue, it can preempt resources from lower-priority tasks, forcing them to pause or migrate.

[0128] This invention achieves adaptive scaling and adjustment of resources across multiple dimensions by constructing a task performance prediction model and a resource importance analysis mechanism, significantly improving cluster resource utilization and task scheduling efficiency. Based on task type matching model prediction and resource importance ranking, it can dynamically reduce the amount of non-critical resource requests while ensuring task performance, effectively reducing resource waste and making cluster resource allocation more aligned with actual task needs, thus improving overall resource utilization. Through task queue sorting, server resource status awareness, and optimized deployment strategies, it can prioritize matching high-priority tasks with high-availability resource nodes, reducing task waiting time and resource fragmentation, enhancing the system's ability to handle real-time, high-concurrency AI tasks, and ultimately achieving intelligent, efficient, and refined cluster resource scheduling. This provides a superior resource scheduling solution for task management in cloud computing, big data, and artificial intelligence scenarios, significantly reducing operating costs.

[0129] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0130] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. An AI computing power platform based on an "elastic scaling" mechanism, characterized in that, It includes a task queue module, an AI computing power scheduling module, a task resource prediction module, a historical operation log database, a status synchronization feedback module, multiple user terminals, and multiple server nodes. The AI ​​computing power scheduling module is connected to the task resource prediction module, the task queue module, the status synchronization feedback module, and the multiple server nodes. The task queue module is also connected to the multiple user terminals, and the task resource prediction module is also connected to the historical operation log database. The task queue module is used to maintain a dynamic task list consisting of user tasks submitted by the multiple user terminals, including original tasks and updated tasks; The historical operation log database is used to collect historical task log data of user tasks in multiple dimensions, including resource usage, execution time, and task type, as the data foundation for the task resource prediction module. The task resource prediction module is used to predict multi-dimensional resource requirements based on the historical operation log data and output the results to the AI ​​computing power scheduling module. The AI ​​computing power scheduling module is used to deploy tasks to server nodes that can meet resource requirements based on task resource prediction results and a preset AI computing power resource scaling scheduling algorithm, thereby realizing dynamic resource scheduling of cluster job resource allocation. The status synchronization feedback module is used to update the task execution status and cluster resource status, and to report the status changes to the task queue module and the AI ​​computing power scheduling module.

2. The AI ​​computing power platform based on the "elastic scaling" mechanism according to claim 1, characterized in that, The task resource prediction module extracts AI computing cluster log data from the historical operation log database, selects m features and n samples of multi-dimensional resource usage from the AI ​​computing cluster log data, and constructs a regression model of task resource requirements under multi-dimensional resources. The feature values ​​of the regression model data include the number of CPU cores, GPU percentage utilization, memory capacity requirements, network traffic, and video memory percentage utilization. The target value of the model data is y. i Let be the execution time of the i-th sample; where the AI ​​computing power cluster log data includes resource data and task type data in multiple dimensions during task execution, and the multiple dimensions of resource data specifically include GPU request amount, CPU request amount, memory request amount, network bandwidth, and IO usage; The XGBoost machine learning method is used for modeling regression. The output expression of the task resource demand prediction model based on XGBoost is: in, t represents the predicted value after t iterations, i.e., the task execution time predicted by the model based on resource conditions; k is the number of decision trees obtained through iteration; and f is the output function of each decision tree. The objective function of the task resource requirement prediction model based on XGBoost is: The task resource requirement model based on XGBoost improves the accuracy of the decision tree model by iterating the residuals of the fitted model t times; where λ and γ are hyperparameters, T represents the number of leaf nodes, g is the first-order partial derivative, h is the second-order partial derivative, and l is the loss function that measures the model fit. The modeling process of the task resource demand prediction model establishes the model structure based on minimizing the objective function value. It then selects the feature value splitting point of the leaf node by maximizing the branch gain, sequentially constructing new branches of the tree model for each feature in the model. Finally, it completes the modeling of the task resource demand XGBoost model. The branch gain is the difference between the objective function value before and after the feature value splitting of the leaf node, expressed as: Each time, the feature with the largest branch gain is selected as the splitting feature of the leaf node. Resource features are selected in turn to complete the construction of the task resource demand prediction model, so that the overall objective function of the model is minimized. Finally, the task resource demand prediction model is established to complete the data fitting prediction.

3. The AI ​​computing power platform based on the "elastic scaling" mechanism according to claim 2, characterized in that, The AI ​​computing power scheduling module defines the problem of multi-dimensional resource scheduling of AI computing power as follows: Suppose there exists an AI computing cluster with a heterogeneous server list S = {S1, S2, ..., S...} with M different resource quotas. M The R-dimensional resource capacity corresponding to the i-th server is represented as follows: The resource request amount corresponding to the j-th user task is The formal definition of the resource allocation and scheduling problem is: Wherein, variable x i,j This indicates that the j-th user task is assigned to the i-th server. During scheduling, tasks can be distributed across multiple servers, so x i,j It can be a fraction; p j It represents the task execution benefit of task j in the current scheduling round, and the scheduling goal is to maximize the task execution benefit in the current scheduling round.

4. The AI ​​computing power platform based on the "elastic scaling" mechanism according to claim 3, characterized in that, In the AI ​​computing power scheduling module, the preset AI computing power resource scaling scheduling algorithm is set according to the problem definition as follows: S1. Extract the tasks that need to be scheduled from the task pool and update the task queue for the current round; wherein, the task queue includes online tasks and tasks to be processed; S2. If there are no tasks in the task queue of the current round, the scheduling of this round will end directly. S2. If there are tasks in the task queue of the current round, sort the task queue of the current round according to the preset scheduling strategy. S3. For each task, an adaptive resource scaling algorithm is used to perform adaptive resource scaling based on the task type, and the resource request quantity is updated and adjusted. S4. Sort the servers according to the amount of available resources from most to least; S5. Check if there are servers that can meet the resource requirements of the task. If so, extract the server set, deploy the task to the server set, and update the server status. If not, put the task into the pending queue and wait for it to be scheduled again in the next scheduling round. S6. Mark the status of the task to update the task queue status, so as to facilitate the next round of scheduling.

5. The AI ​​computing power platform based on the "elastic scaling" mechanism according to claim 4, characterized in that, In step S3, the preset adaptive resource scaling algorithm includes: S301, According to task type C j Load the task resource requirement prediction model; S302. Calculate the amount of resources w requested by the current user. j Expected performance of the task t j ; S303. Calculate the importance of each resource type to task performance based on the task resource requirement prediction model, and generate an importance list; S304. When the adaptive adjustment rate α < 1, for each resource r, if r is not on the importance list, its application quantity is reduced proportionally to α. S305. After adjustment, calculate the new resource W. j Performance T j ; S306, Calculation error = |T j -t j If error < θ, then return the adjusted resource W. j Otherwise, increase α, i.e., α = α + Δα, and try to reduce more non-critical resources again. S307. When the adaptive adjustment rate α > 1, if the error requirement still cannot be met, then abandon scaling and return to the original resource request amount w. j .

6. The AI ​​computing power platform based on the "elastic scaling" mechanism according to claim 1, characterized in that, It also includes a resource monitoring module, which is connected to the AI ​​computing power scheduling module. The resource monitoring module is used to collect multi-dimensional resource data from multiple server nodes in real time to generate a resource topology map and to perform anomaly detection and analysis on the resource data. The multi-dimensional resource data includes CPU utilization, GPU memory usage, memory usage, network bandwidth, and storage I / O.

7. The AI ​​computing power platform based on the "elastic scaling" mechanism according to claim 6, characterized in that, The resource monitoring module performs anomaly detection and analysis on resource data, including anomaly detection based on preset rules and anomaly detection based on machine learning. Anomaly detection based on preset rules includes: If CPU utilization is set to be greater than a first set value and remains so for a first set duration, it is marked as node overload; if GPU memory usage is set to be less than a second set value and the task queue is backlogged, it is marked as resource waste. If CPU utilization is set to be greater than the third setting and network bandwidth is set to be less than the fourth setting, then a computing-network bottleneck is identified, and task migration is triggered. The sliding window mean of the metric is calculated and compared with the historical average. If the difference exceeds the first preset threshold, it is marked as abnormal. Machine learning-based anomaly detection includes: An autoencoder is used to train a model to reconstruct normal resource data. Samples with reconstruction errors exceeding a second preset threshold are judged as abnormal.

8. The AI ​​computing power platform based on the "elastic scaling" mechanism according to claim 1, characterized in that, It also includes a task priority management module, which is connected to the status synchronization and feedback module. The task priority management module is used to assign priority tags to tasks and prioritize high-priority tasks during scheduling. The methods for determining task priorities include: directly defining priorities by the tags, types, and user levels provided when the task is submitted; dynamically adjusting priorities based on resource utilization, execution progress, and dependencies during task execution; or manually adjusting task priorities by administrators or users.

Images