Cloud aggregation AI model automatic arrangement system

Abstract

The invention relates to the technical field of model deployment, in particular to a cloud aggregation AI model automatic arrangement system which comprises a task scheduling module, a compatibility analysis module, a framework matching module, a deployment optimization module and a scheduling intervention module. According to the method, the real-time load condition of the platform, the calculation requirements of the tasks and the pre-estimated execution time are comprehensively evaluated before task arrangement, and the execution sequence with the priority is dynamically constructed, so that priority processing of key tasks is ensured, and the overall utilization efficiency of calculation resources is improved; resource congestion and scheduling delay caused by simple queuing execution of tasks are avoided, success rate, error rate and performance data in historical execution records of the tasks are deeply analyzed in terms of deployment framework selection, compatibility judgment is formed by combining real-time performance feedback at the initial stage of deployment, and the task scheduling efficiency is improved. Furthermore, the structural features of the AI model are matched with the underlying capability of the frame, and the transformation from coarse-grained resource linkage to fine capability adaptation is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of model deployment, in particular to a cloud computing AI model automatic arrangement system. BACKGROUND

[0002] The cloud computing technology field covers the core technologies of computer resource virtualization, scheduling and management, especially the centralized management and on-demand allocation of computing, storage, network and other hardware resources through the network.

[0003] Among them, the cloud computing AI model automatic arrangement system refers to the centralized scheduling and execution of multiple AI model deployment tasks, realizing the automatic arrangement and linkage of each stage task from resource preparation to online operation of the model.

[0004] The prior art has significant limitations in centralized scheduling and automatic arrangement mechanism when dealing with diversified AI model deployment tasks. In actual operation, due to the lack of fine-grained evaluation of task priority and resource matching degree, when multiple model tasks with large differences in computing demand and execution time are concurrent, simple arrangement and linkage cannot effectively allocate resources. For example, a long-time-consuming low-importance analysis task may occupy computing resources for a long time, thereby blocking the execution of high-importance real-time inference tasks that require fast response, resulting in low overall processing efficiency and service delay. At the same time, the technology does not consider the internal compatibility and capability adaptability between the model and the deployment framework before deployment, only performs general resource preparation and linkage. This approach often leads to model deployment failure due to incompatibility of underlying operators, or performance bottlenecks and resource waste due to poor memory management mechanism of the framework during runtime. Finally, after the technology is deployed, it is suspended, lacking the ability of continuous tracking and dynamic optimization of the running state, and cannot cope with actual situations such as platform load fluctuation or task performance degradation, seriously affecting the stability and reliability of AI model services. SUMMARY

[0005] The purpose of the present application is to solve the shortcomings in the prior art and to provide a cloud computing AI model automatic arrangement system.

[0006] In order to achieve the above purpose, the present application adopts the following technical scheme: a cloud computing AI model automatic arrangement system, the system comprises: A task scheduling module acquires the computing resource information of the cloud computing platform, determines the load condition of the current platform according to the computing resource information, adjusts the task execution order by referring to the computing demand and execution time of each task, and constructs an initial arrangement task. A compatibility analysis module acquires the task execution records of each type of task running on all deployment frameworks, and the real-time feedback information corresponding to the initial deployment stage or the test stage, and constructs a framework compatibility list. A framework matching module corresponds the framework compatibility list with the computing requirement and execution time of each task in the initial task arrangement, analyzes the capability adaptability between the AI model and the framework, and constructs a deployment adaptation list; A deployment optimization module selects the initial deployment framework of each task according to the deployment adaptation list, dynamically optimizes in combination with the inference rate of task runtime and the platform load condition in the deployment stage, and generates a task arrangement optimization result.

[0007] The initial task arrangement is specifically task execution order, computing resource load condition, task computing requirement and task execution time, the framework compatibility list includes task execution record, real-time feedback information and framework compatibility, the deployment adaptation list is specifically task framework adaptability analysis, task computing requirement matching and task execution time matching, and the task arrangement optimization result is specifically task optimization order, inference rate adjustment, platform load adjustment and deployment framework selection.

[0008] The task scheduling module comprises: A resource load acquisition submodule acquires computing resource information of the cloud aggregation platform, including CPU core number, GPU display memory and memory resource usage of each node, obtains node usage rate by calculating the proportion of resource usage of each node to total resource amount, and determines platform load information; A task requirement matching submodule acquires computing requirement and estimated execution time of each task, compares the computing requirement of each task with the current available resource of the platform node, determines the matching degree of the computing requirement and the platform load information, establishes execution priority of each task, and adjusts the position of each task in the dynamic queue with the execution priority and task estimated execution time of each task as a reference to obtain an initial task arrangement sequence.

[0009] The compatibility analysis module comprises: An execution record extraction submodule acquires task execution record of each type of task when running on all deployment frameworks and performs normalization processing, calculates execution record compatibility, and the task execution record includes execution success rate, error rate, inference time and memory occupation; A real-time feedback analysis submodule acquires real-time feedback information corresponding to the initial deployment stage or the test stage, acquires performance feedback and compatibility performance of the current task on each framework, performs numerical adjustment based on the execution record compatibility and the performance in the real-time feedback, and generates a dynamic compatibility evaluation result; ​The compatibility list construction submodule compares the dynamic compatibility evaluation results between each task and each candidate deployment framework with a preset compatibility matching benchmark, filters deployment frameworks with a matching degree higher than the benchmark, and sorts and constructs a framework compatibility list.

[0010] The framework matching module comprises: The task feature extraction submodule corresponds the calculation requirement, execution time and AI model structure feature of each task, integrates task attributes and framework information, and obtains a task framework feature set by calling the framework compatibility list and the initial orchestration task. The capability adaptation evaluation submodule compares the number of layers and parameter size of each AI model with the scheduling delay and memory management capability of the candidate framework item by item based on the task framework feature set, and obtains a capability adaptation result. The deployment list generation submodule constructs a deployment adaptation list according to the capability adaptation result of each task and the current cloud computing platform candidate framework sorting.

[0011] The deployment optimization module comprises: The initial deployment selection submodule corresponds each task with the deployment framework ranked first after evaluation according to the deployment adaptation list, establishes a mapping relationship between the task and the framework, and obtains a task initial deployment result. The dynamic optimization adjustment submodule monitors the inference rate of each task in runtime and the real-time load condition of the cloud computing platform based on the task initial deployment result, marks the task with an inference rate lower than a preset inference rate threshold when the load is higher than a preset load threshold, and generates dynamic state information. The task orchestration generation submodule calls the dynamic state information of each task, rearranges the execution order of the task marked as needing adjustment in combination with the original execution priority, obtains a task orchestration optimization result, and is used for dynamic task deployment optimization of automatic orchestration of the cloud computing AI model.

[0012] The scheduling intervention module comprises: The task orchestration intervention information comprises resource prediction, task scheduling intervention, scheduling congestion prediction and resource shortage prediction.

[0013] The scheduling intervention module comprises: A running parameter monitoring submodule continuously monitors the running parameters of each task based on the task arrangement optimization result, including CPU occupancy, GPU load, memory peak value, and storage I / O, compares them with preset upper and lower limits of each running threshold, quantifies the deviation degree, and obtains resource running state information; A resource trend prediction submodule collects the resource running state information in a continuous time window, extrapolates the change trend of the resource running state, identifies the rate of resource usage growth over time, predicts future resource shortage sections or scheduling congestion points, and generates scheduling congestion risk information; An arrangement intervention generation submodule obtains the predicted congestion node and task corresponding to the scheduling congestion risk information, and establishes task arrangement intervention information.

[0014] Compared with the prior art, the advantages and positive effects of the present application are that: In the present application, by comprehensively evaluating the real-time load status of the platform, the computing demand and the estimated execution time of the task before task arrangement, a priority execution sequence is dynamically constructed, ensuring the priority processing of critical tasks and improving the overall utilization efficiency of computing resources, avoiding resource congestion and scheduling delay caused by simple queuing execution of tasks, and in the selection of deployment framework, the success rate, error rate and performance data in the historical execution record of the task are analyzed in depth, combined with real-time performance feedback at the beginning of deployment to form compatibility judgment, further matching the structural features of the AI model with the underlying capabilities of the framework, realizing the transformation from coarse-grained resource linkage to fine-grained capability adaptation, greatly reducing the risk of deployment failure and ensuring the optimal performance of model running. At the same time, the arrangement process extends to continuous monitoring and dynamic adjustment after deployment, which can re-optimize the task order according to the real-time changes of inference rate and platform load, even predict future resource shortage conditions through trend analysis and generate intervention information, so that the entire arrangement process has the closed-loop optimization capability from passive execution to active intervention, effectively ensuring the high stability and continuous efficient operation of AI model services in cloud computing environment. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 The system module diagram of the present application; Figure 2 The system framework diagram of the present application; Figure 3 The schematic diagram of the task scheduling module of the present application; Figure 4 The schematic diagram of the compatibility analysis module of the present application; Figure 5 The schematic diagram of the framework matching module of the present application; Figure 6 The schematic diagram of the deployment optimization module of the present application; Figure 7 A schematic diagram of a scheduling intervention module of the present application. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and do not limit the present application.

[0017] Referring to Figure 1 The present application provides a technical solution: a cloud computing algorithm AI model automatic arrangement system, which comprises: A task scheduling module acquires computing resource information of a cloud computing algorithm platform, determines the load condition of the current platform according to the computing resource information, adjusts the task execution order in reference to the computing demand and execution time of each task, and constructs an initial arrangement task; A compatibility analysis module acquires task execution records of each type of task when running on all deployment frameworks, and real-time feedback information corresponding to the initial deployment stage or the test stage, and constructs a framework compatibility list; A framework matching module corresponds the framework compatibility list to the computing demand and execution time of each task in the initial arrangement task, analyzes the capability adaptability between the AI model and the framework, and constructs a deployment adaptation list; A deployment optimization module selects the initial deployment framework of each task according to the deployment adaptation list, and dynamically optimizes in combination with the inference rate of the task when running and the platform load condition in the deployment stage to generate a task arrangement optimization result; The initial arrangement task is specifically the task execution order, the computing resource load condition, the task computing demand, and the task execution time. The framework compatibility list includes the task execution record, the real-time feedback information, and the framework compatibility. The deployment adaptation list is specifically the task framework adaptability analysis, the task computing demand matching, and the task execution time matching. The task arrangement optimization result is specifically the task optimization order, the inference rate adjustment, the platform load adjustment, and the deployment framework selection.

[0018] Referring to Figure 2 and Figure 3 The task scheduling module comprises: A resource load acquisition sub-module acquires the computing resource information of the cloud computing algorithm platform, including the CPU core number, GPU display memory, and memory resource usage of each node, obtains the node usage rate by calculating the proportion of the resource usage of each node to the total resource amount, and determines the platform load information; Firstly, through the dedicated monitoring agent program, the total amount and real-time usage of CPU core number, GPU memory, and memory resources of each computing node in the cloud aggregation platform are obtained, and are collected at a frequency of seconds. Then, the CPU usage rate, GPU usage rate, and memory usage rate of the node are obtained by calculating the ratio of the usage of each resource on the node to the total resource amount of the node. For example, the total CPU core number of a node is 16, and the current number of cores in use is 12. The CPU usage rate is obtained by calculating 12 divided by 16. Finally, the arithmetic mean of the CPU, GPU, and memory usage rates of all nodes is obtained as the overall load information of the platform.

[0019] The task demand matching sub-module collects the computing demand and estimated execution time of each task, compares the computing demand of each task with the current available resources of the platform node, determines the matching degree of the computing demand and the platform load information, and establishes the execution priority of each task. First, the computing demand of each to-be-processed task is collected from the task queue, including the required CPU core number , GPU memory , and memory , and the estimated execution time of the task . Then, the real-time available resource information of each computing node of the platform provided by the previous resource load collection sub-module is called, including the available CPU core number , available GPU memory , and available memory . In order to determine the occupation degree of the platform resources by the task, the sub-module calculates the “demand proportion” of the task in each resource dimension, that is, the ratio of the demand amount to the available amount of the node. In order to unify the dimension, the demand of the three dimensions is converted to a dimensionless value in the form of a ratio. The resource bottleneck of the task depends on the highest demand proportion of one resource, therefore, a “maximum resource occupation rate” is defined to directly measure the pressure of the task on the platform. The calculation formula is: The advantage of this formula is that it captures the short-board resource that determines whether the task can run efficiently by taking the maximum value. Finally, based on the maximum resource occupation rate, the sub-module establishes the execution priority of each task combined with the execution time of the task, and the calculation formula is: The advantage of this formula is that its structure clearly expresses the inverse relationship between priority and resource occupation and time cost. The two product terms in the denominator represent resource pressure and time pressure, respectively, and the numerator is 1, which establishes a benchmark, so that the greater the pressure, the lower the priority value. The explanations of the parameters in the formula are as follows: is the final execution priority value calculated for the current task. is the maximum resource occupancy, which represents the resource bottleneck of the task. , , are the CPU core number, GPU memory (GB) and memory (GB) required by the task, respectively. , , are the CPU core number, GPU memory (GB) and memory (GB) currently available on the computing node, respectively. is the function that takes the maximum value. is the time penalty coefficient, which is set according to the sensitivity of the platform to the task turnaround time. The more stringent the platform's requirements for high turnaround rate, the greater the value should be, thereby increasing the priority penalty for long-time-consuming tasks. is the estimated execution time of the current task in seconds. is the average execution time obtained by statistical history of similar tasks in seconds, used for normalization of to make it a dimensionless ratio reflecting the relative time cost.

[0020] Take "image recognition task A" as an example for calculation. First, calculate its maximum resource occupancy , set the parameter values: , , . , , .

[0021] Calculate the demand proportion of each resource dimension: CPU demand proportion: GPU memory demand proportion: Memory demand proportion: .

[0022] Substitute the formula to calculate .

[0023] Next, use the calculated to calculate the final execution priority . Set the remaining parameter values: , indicating a moderate degree of punishment for long-time-consuming tasks. seconds. seconds.

[0024] Substitute the above values into the execution priority formula: . The final task execution priority is 0.2007 The task scheduling construction submodule calls the execution priority of each task and the estimated execution time of the task, takes the scheduling priority benchmark as a reference, adjusts the position of each task in the dynamic queue, and obtains an initial task scheduling queue; First, the execution priority value of each task is called, then a fixed scheduling priority benchmark is taken as a reference, the benchmark value is set to 0.5, which is determined by statistical analysis of the historical task priority distribution of the platform, and is used to distinguish the relative urgency of the task, then a high priority task list and a low priority task list are created, all tasks to be scheduled are traversed, the execution priority of each task is compared with the benchmark, if the priority value of the task is higher than the benchmark, the task is added to the high priority task list, otherwise it is added to the low priority task list, in both lists, the execution priority value of each task is arranged in descending order, finally, the high priority task list and the low priority task list are spliced to form the final initial task scheduling queue.

[0025] Please refer to Figure 2 and Figure 4 , the compatibility analysis module includes: The execution record extraction submodule obtains the task execution record of each type of task running on all deployment frameworks and performs normalization processing, calculates the execution record compatibility, and the task execution record includes execution success rate, error rate, inference time and memory occupation; First, the execution record of the specified task is called from the historical execution log database of the cloud aggregation algorithm platform, including execution success rate, error rate, inference time and memory occupation, then the normalized indicators are multiplied by their corresponding weights and then weighted summed to calculate the execution record compatibility of the task on the framework , the calculation formula is: The advantage of this formula is that it directly integrates the normalization process into the compatibility calculation, and through positive and negative weights, it clearly shows the contribution direction of each indicator to the compatibility, making the evaluation model more intuitive and interpretable, the explanations of the parameters in the formula are as follows: is the execution record compatibility value calculated for the current task and framework combination; is the original value of the execution success rate extracted from the execution record; is the original value of the error rate extracted from the execution record; is the original value of the inference time extracted from the execution record, in milliseconds; is the original value of the memory occupation extracted from the execution record, in megabytes; , , , is the maximum value obtained by statistical analysis of the corresponding indicators of all tasks of the same type in the historical database, which has the same dimension as the original value, and is used to convert the original measurement values of different dimensions into dimensionless ratios between 0 and 1; is the weight of the execution success rate, which is set according to the basic importance of task success execution. Task success is the premise of all performance indicators, so it is the most important and is given the largest positive weight; is the weight of the error rate, which is set according to the direct negation of errors to the value of the task. The higher the error rate, the greater the possibility of task failure, and the extremely significant negative impact, so it is given a negative weight with a larger absolute value; is the weight of the inference time, which is set according to the pursuit of high throughput and fast response by the platform. The longer the inference time, the lower the performance, so it is given a negative weight, and the higher the requirement of the platform for performance, the larger the absolute value of the weight; is the weight of the memory occupation, which is set according to the finiteness of the platform resources. The higher the memory occupation, the fewer the number of tasks that can be executed concurrently, and the more intense the resource competition, so it is given a negative weight, and the more tense the platform memory resources, the larger the absolute value of the weight.

[0026] Assume that the execution of a certain image recognition task on framework X is evaluated, and the values of each parameter are set as follows: execution success rate weight 0.6; error rate weight 0.2; inference time weight 0.1; memory occupation weight 0.1. The original values extracted from the execution record are as follows: execution success rate 0.98; error rate 0.02; inference time 150 milliseconds; memory occupation 2048 megabytes. The maximum values obtained by statistical analysis of historical tasks of the same type are as follows: maximum success rate 1.0; maximum error rate 1.0; maximum inference time 1000 milliseconds; maximum memory occupation 8192 megabytes. Substitute the above values into the formula to calculate: .

[0027] The real-time feedback analysis submodule collects real-time feedback information corresponding to the initial deployment stage or the test stage, obtains the performance feedback and compatibility performance of the current task on each framework, and adjusts the values based on the performance in the execution record compatibility and real-time feedback to generate a dynamic compatibility evaluation result; During the initial 100 calls of task deployment or the test run lasting 5 minutes, real-time performance feedback of the current task on each candidate framework is collected, the average inference success rate of the initial stage is obtained, then the execution record compatibility of the task on the framework is called, and then the historical data and real-time data are weighted and fused according to the preset dynamic adjustment coefficient. The specific calculation process is: multiplying the execution record compatibility by its corresponding weight, multiplying the average inference success rate collected in real time by its corresponding weight, and finally adding the two products. The weight is set according to the fact that, in the absence of a large amount of real-time data, the historical data verified for a long time have more reference value, so they are given a higher weight, for example, 0.7, while the real-time data weight is 0.3, thereby generating a dynamic compatibility evaluation result.

[0028] The compatibility list construction submodule compares the dynamic compatibility evaluation result between each task and each candidate deployment framework with a preset compatibility matching benchmark, filters deployment frameworks with a matching degree higher than the benchmark and sorts them, and constructs a framework compatibility list. First, a preset compatibility matching benchmark is called, which is set to 0.85. The basis for setting this benchmark is to statistically analyze the dynamic compatibility evaluation results of all successfully deployed task and framework combinations in the historical data, and take the value of the 85th percentile to ensure that only excellent frameworks are included in the candidates. Subsequently, an empty compatible framework list is initialized, all candidate deployment frameworks are traversed, the dynamic compatibility evaluation result of each framework is compared with the compatibility matching benchmark in terms of numerical size, and if the evaluation result is higher than the benchmark, the framework and its evaluation result are added as an entry to the compatible framework list. After all the frameworks are traversed, the list is sorted in descending order according to the evaluation result value to construct the final framework compatibility list.

[0029] Please refer to Figure 2 and Figure 5 , the framework matching module includes: The task feature extraction submodule calls the framework compatibility list and the initial arrangement task, matches the computing demand, execution time and AI model structure characteristics of each task, integrates the task attributes and framework information, and obtains the task framework feature set. First, the metadata of the task is accessed to extract its computing demand, including the specific values of CPU core number, GPU memory and memory. Then, the structure definition file of the AI model is parsed to extract the structural characteristics of the model, such as the total number of layers, the number of various operators and the parameter size. Finally, the two parts of information are integrated with the attribute information of each framework in the framework compatibility list to form a structured data object containing task computing demand, model structure characteristics and framework support characteristics, and the task framework feature set is obtained.

[0030] The capability adaptation evaluation submodule compares the number of layers, parameter size, scheduling delay, and memory management capability of each AI model with the candidate framework based on the task framework feature set to obtain a capability adaptation result. The capability adaptation evaluation submodule compares the number of layers, parameter size, scheduling delay, and memory management capability of each AI model with the candidate framework based on the task framework feature set to obtain a capability adaptation result.

[0031] The deployment list generation submodule constructs a deployment adaptation list according to the capability adaptation result of each task and the current cloud aggregation platform candidate framework ranking. First, a candidate deployment list is created for each task, and the initial content of the list is the framework compatibility list. Then, the capability adaptation result of each task and each framework is used as the main sorting basis for the list, and the list is sorted in descending order. If there are two frameworks with the same capability adaptation result value, the original ranking of the frameworks in the framework compatibility list is used as the secondary sorting basis. The framework with a higher ranking is preferred. Finally, a deployment adaptation list optimized by double sorting is generated for each task.

[0032] Please refer to Figure 2 and Figure 6 The deployment optimization module includes: The initial deployment selection submodule maps each task to the deployment framework ranked first after evaluation according to the deployment adaptation list to obtain the initial deployment result of the task. All tasks to be deployed are traversed. For each task, such as "image recognition task A", the submodule accesses the corresponding deployment adaptation list and directly selects the deployment framework ranked first in the list, such as "framework X". Then, the submodule establishes a clear deployment mapping relationship between the task and the selected framework. This mapping process is specifically manifested as follows: in a newly created deployment configuration set, an entry is created, which binds the unique identifier of the task (such as the ID of "image recognition task A") with the unique identifier of the selected framework (such as the ID of "framework X") to form a key-value pair. This operation is repeated for each task in the deployment adaptation list until all tasks have been assigned to their optimal initial deployment framework. Finally, all these mapping relationships collectively constitute the complete initial deployment result of the task, which will be used as the direct basis for subsequent deployment operations.

[0033] The dynamic optimization adjustment submodule monitors the inference rate of each task at runtime and the real-time load of the cloud computing platform based on the initial deployment results of the tasks. When the load is higher than a preset load threshold, the tasks with an inference rate lower than a preset inference rate threshold are marked, and dynamic state information is generated. After the tasks are deployed, the inference rate of each task at runtime and the real-time load of the cloud computing platform are continuously monitored. When it is monitored that the platform load is higher than a preset load threshold of 85%, which is set based on the 90th percentile of the daily peak load of the platform in the past month to avoid false judgments under extremely high load, the system checks the inference rate of the currently running tasks, and marks the tasks with an inference rate lower than a preset inference rate threshold as tasks to be adjusted. The inference rate threshold is set according to the minimum performance standard promised by the task type in the service level agreement (SLA) or the 80% historical average inference rate of similar tasks under standard hardware configuration. Dynamic state information is generated.

[0034] The task orchestration generation submodule calls the dynamic state information of each task, and rearranges the execution order of the tasks marked as tasks to be adjusted in combination with the original execution priority, to obtain task orchestration optimization results for dynamic task deployment optimization of automatic orchestration of the cloud computing AI model. First, all tasks marked as to be adjusted are selected from the dynamic state information to form a to-be-optimized set. Then, the execution order of the tasks in the set is rearranged in combination with the original execution priority determined for the tasks in the task demand matching submodule. Specifically, within the to-be-adjusted task set, the tasks with high original execution priority are moved forward in the dynamic queue, and the tasks with low priority are moved backward, to obtain task orchestration optimization results.

[0035] Please refer to Figure 2 and Figure 7 The scheduling intervention module also includes a scheduling intervention module that continuously monitors the running parameters of each task based on the optimized task list, sets upper and lower limits of running thresholds, predicts future resource shortage sections or scheduling congestion points, and generates task orchestration intervention information. The task orchestration intervention information includes resource prediction, task scheduling intervention, scheduling congestion prediction, and resource shortage prediction. The scheduling intervention module includes: The running parameter monitoring submodule continuously monitors the running parameters of each task based on the task orchestration optimization results, including CPU occupancy, GPU load, memory peak, and storage I / O, compares them with preset upper and lower limits of each running threshold, quantifies the deviation, and obtains resource running state information. Based on the task arrangement optimization result, during the task execution, the CPU occupancy rate, GPU load, memory peak value, storage I / O and other running parameters of each task are monitored at a period of 1 second, and these real-time parameters are compared with the preset upper limit of each running threshold value, for example, the upper limit threshold value of CPU occupancy rate is 90%, which is set according to the stable running condition parameters provided by the node hardware manufacturer and leaving a safety margin of 10%, then, by calculating the difference between the real-time value and the threshold upper limit, and then dividing the difference by the threshold upper limit, the quantitative deviation degree is obtained, and the resource running state information is obtained.

[0036] The resource trend prediction sub-module collects the resource running state information in the continuous time window, and extrapolates the change trend of the resource running state, identifies the rate of resource usage growth over time, predicts future resource shortage sections or scheduling congestion points, and generates scheduling congestion risk information. First, the resource running state information of each second in the past 300 seconds is collected to form a time series data set containing 300 data points, where each data point is a timestamp and the corresponding resource deviation degree, then, the linear regression algorithm is applied to the data set to calculate the best fitting straight line equation describing the trend of the deviation degree over time , where represents the deviation degree, represents time, (slope) represents the growth rate of the deviation degree, (intercept) is the initial deviation degree, then, the equation is used for prediction, by setting the target deviation degree as the preset congestion risk threshold value of 20%, the equation is solved to obtain the time , so as to predict the specific future time point when the resource deviation degree will reach the threshold value, the setting of the threshold value is based on historical operation and maintenance data analysis, which shows that when the resource deviation degree exceeds 20%, the risk of task scheduling failure or node crash of the platform significantly increases, and the scheduling congestion risk information is generated.

[0037] The arrangement intervention generation sub-module obtains the predicted congestion node and task corresponding to the scheduling congestion risk information, and establishes task arrangement intervention information. First, the scheduling congestion risk information generated by the aforementioned resource trend prediction submodule is obtained, which explicitly indicates which specific computing node will reach the congestion risk threshold at a specific time point in the future. According to the node identifier in the information, the submodule immediately queries the current state of the computing node, traverses all running tasks on the node, and calls the resource running state information of each task in the latest monitoring period, especially the growth rate of each task's deviation from various resources (such as CPU and GPU load). By comparing the values of these growth rates, the submodule locates the task that causes the fastest growth of resource consumption and identifies it as the key task that triggers the congestion risk. Subsequently, the submodule integrates and packages multiple key information pieces, including the unique identifier of the computing node predicted to experience congestion, the task identifier identified as the key task, the specific resource type (such as CPU occupancy) facing the risk, and the remaining time to reach the congestion threshold. These information are combined into a structured and explicit instruction object, namely task orchestration intervention information. This information will be passed to the subsequent processing flow as the direct basis for triggering specific intervention actions (such as task migration or resource limitation), thereby actively avoiding congestion before it actually occurs.

[0038] The above is only a preferred embodiment of the present application, and does not limit the present application in other forms. Any skilled person in the art can use the disclosed technical content to make changes or modifications to equivalent embodiments applied to other fields, but any simple modification, equivalent change and modification made in accordance with the technical essence of the present application to the above embodiments without departing from the technical solution content of the present application shall still fall within the protection scope of the present application.

Claims

1. A cloud aggregation algorithm AI model automatic arrangement system, characterized in that: The system comprises: a task scheduling module, which acquires computing resource information of a cloud computing platform, determines a current platform load condition according to the computing resource information, adjusts a task execution order in reference to a computing demand and an execution time of each task, and constructs an initial task arrangement; a compatibility analysis module, which acquires task execution records of each type of task when running on all deployment frameworks, and real-time feedback information corresponding to an initial deployment stage or a test stage, and constructs a framework compatibility list; a framework matching module, which corresponds the framework compatibility list to the computing demand and the execution time of each task in the initial task arrangement, analyzes capability adaptability between an AI model and a framework, and constructs a deployment adaptation list; a deployment optimization module, which selects an initial deployment framework for each task according to the deployment adaptation list, and dynamically optimizes in combination with an inference rate of a task when running and a platform load condition in a deployment stage, to generate a task arrangement optimization result.

2. The cloud cluster AI model automatic arrangement system according to claim 1, characterized in that: The initial task arrangement specifically refers to a task execution order, a computing resource load condition, a task computing demand, and a task execution time, the framework compatibility list includes task execution records, real-time feedback information, and framework compatibility, the deployment adaptation list specifically refers to task framework adaptability analysis, task computing demand matching, and task execution time matching, and the task arrangement optimization result specifically refers to a task optimization order, an inference rate adjustment, a platform load adjustment, and a deployment framework selection. 3.The cloud cluster AI model automatic arrangement system according to claim 1, characterized in that: The task scheduling module comprises: a resource load acquisition submodule, which acquires computing resource information of a cloud computing platform, including CPU core numbers, GPU display memories, and memory resource usage of each node, obtains a node usage rate by calculating a proportion of resource usage of each node to total resource quantity, and determines platform load information; a task demand matching submodule, which acquires a computing demand and an estimated execution time of each task, compares the computing demand of each task with currently available resources of a platform node, determines a matching degree of the computing demand to the platform load information, and establishes an execution priority of each task; a task arrangement construction submodule, which calls the execution priority of each task and the task estimated execution time, takes a scheduling priority benchmark as a reference, adjusts a position of each task in a dynamic queue, and obtains an initial task arrangement order.

4. The cloud cluster AI model automatic arrangement system according to claim 3, characterized in that: Execution priority for each task using the formula: ； wherein, is the maximum resource occupancy rate, is the estimated execution time of the current task, is the average execution time obtained by statistics of historical similar tasks, is the time penalty coefficient. 5.The cloud cluster AI model automatic arrangement system according to claim 1, characterized in that: The compatibility analysis module comprises: an execution record extraction submodule, which acquires task execution records of each type of task when running on all deployment frameworks and performs normalization processing, calculates an execution record compatibility degree, and the task execution records include an execution success rate, an error rate, an inference time, and a memory occupation; a real-time feedback analysis submodule, which acquires real-time feedback information corresponding to an initial deployment stage or a test stage, acquires performance feedback and compatibility performance of a current task on each framework, performs numerical adjustment based on the execution record compatibility degree and the performance feedback in the real-time feedback, and generates a dynamic compatibility evaluation result; a compatibility list construction submodule, which compares, according to the dynamic compatibility evaluation result between each task and each candidate deployment framework, the dynamic compatibility evaluation result to a preset compatibility matching benchmark, screens deployment frameworks with a matching degree higher than the benchmark and sorts them, and constructs a framework compatibility list. 6.The cloud cluster AI model automatic arrangement system according to claim 5, characterized in that: For computing the record compatibility degree using the formula: ； wherein, is an execution success rate extracted from the execution record, is an error rate extracted from the execution record, is an inference time extracted from the execution record, is a memory occupancy raw value extracted from the execution record, is a maximum value of is a maximum value of is a maximum value of is a maximum value of is a weight of the execution success rate, is a weight of the error rate, is a weight of the inference time, is a weight of the memory occupancy.

7. The cloud cluster AI model automatic arrangement system according to claim 1, characterized in that: The framework matching module comprises: The task feature extraction submodule corresponds the calculation requirement, execution time and structural feature of the AI model of each task based on the framework compatibility list and the initial arrangement task, integrates the task attribute and framework information, and obtains a task framework feature set; The capability adaptation evaluation submodule compares the number of layers, parameter size of each AI model and the scheduling delay and memory management capability of the candidate framework one by one based on the task framework feature set, and obtains a capability adaptation result; The deployment list generation submodule constructs a deployment adaptation list according to the capability adaptation result of each task and the current cloud computing platform candidate framework ranking. 8.The cloud cluster AI model automatic arrangement system according to claim 1, wherein: The deployment optimization module comprises: The initial deployment selection submodule corresponds each task and the deployment framework ranked first after evaluation based on the deployment adaptation list, establishes a mapping relationship between the task and the framework, and obtains a task initial deployment result; The dynamic optimization adjustment submodule monitors the inference rate of each task and the real-time load condition of the cloud computing platform based on the task initial deployment result, marks the task with an inference rate lower than a preset inference rate threshold when the load is higher than a preset load threshold, and generates dynamic state information; The task arrangement generation submodule calls the dynamic state information of each task, rearranges the execution order of the task marked as needing adjustment in combination with the original execution priority, obtains a task arrangement optimization result, and uses the result for dynamic task deployment optimization of the cloud computing AI model automatic arrangement. 9.The cloud cluster AI model automatic arrangement system according to claim 1, wherein: The scheduling intervention module comprises: The task arrangement intervention information comprises resource prediction, task scheduling intervention, scheduling congestion prediction and resource shortage prediction. 10.The cloud cluster AI model automatic arrangement system according to claim 9, characterized in that: The scheduling intervention module comprises: The running parameter monitoring submodule continuously monitors the running parameter of each task based on the task arrangement optimization result, compares the running parameter with a preset upper and lower limit of each running threshold, quantifies the deviation degree, and obtains resource running state information, wherein the running parameter comprises CPU occupancy, GPU load, memory peak value and storage I / O; The resource trend prediction submodule collects the resource running state information in a continuous time window, extrapolates the change trend of the resource running state, identifies the rate of resource usage growth over time, predicts the future resource shortage section or scheduling congestion point, and generates scheduling congestion risk information; The arrangement intervention generation submodule obtains the predicted congestion node and task corresponding to the scheduling congestion risk information, and establishes task arrangement intervention information.

Images