AI-based computing power scheduling method and device, and storage medium

Abstract

The present application relates to the technical field of computing power scheduling, and more particularly to a computing power scheduling method and device based on AI and a storage medium. The method comprises: receiving and analyzing a computing power scheduling request to construct a task execution logical graph; generating a resource demand curve based on the task execution logical graph and a historical experience library; constructing a scheme generation model based on the resource demand curve to generate a scheduling scheme; executing scheduling according to the scheduling scheme and adjusting in a deviation-driven manner during execution; and accumulating experience and updating the historical experience library after the task is completed. By analyzing the internal execution logic and dependency of the task, the present application can predict the differentiated demand curve of various computing power resources at different stages of the entire life cycle of the task in the future, and construct a phased scheduling scheme combining resource reservation and dynamic adjustment.

Description

Technical Field

[0001] This invention relates to the field of computing power scheduling technology, and in particular to an AI-based computing power scheduling method, apparatus and storage medium. Background Technology

[0002] As the scale and complexity of artificial intelligence models continue to grow, computing resources have become a critical infrastructure supporting AI training and inference. Heterogeneous computing nodes (such as GPUs, CPUs, and NPUs) are widely deployed in data centers, and computing power scheduling needs to cope with challenges such as diverse task types, large fluctuations in resource requirements, and dynamic changes in the execution environment. Therefore, a scheduling mechanism with adaptive and intelligent decision-making capabilities is urgently needed.

[0003] Existing computing power scheduling methods mostly employ static resource allocation or simple rule-based scheduling strategies, which struggle to cope with the dynamic changes in resource demands during AI task execution. On one hand, scheduling systems lack refined modeling of subtask dependencies and resource preferences within a task, leading to significant discrepancies between resource reservations and actual needs. On the other hand, existing methods lack a deviation-driven adjustment mechanism based on real-time feedback during execution, failing to respond quickly when task execution exceeds expectations or resources are overestimated. Furthermore, historical execution experience has not been effectively accumulated and used to inform scheduling decisions, resulting in a lack of continuous optimization capabilities for scheduling strategies and limiting overall scheduling efficiency and resource utilization. Summary of the Invention

[0004] The purpose of this invention is to provide an AI-based computing power scheduling method, apparatus, and storage medium to solve at least one of the problems existing in the prior art.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] An AI-based computing power scheduling method includes:

[0007] Receive and parse computing power scheduling requests to construct a task execution logic graph;

[0008] Based on the task execution logic graph and historical experience database, a resource requirement curve is generated.

[0009] A scheme generation model is constructed based on the resource demand curve to generate scheduling schemes;

[0010] Execute scheduling according to the scheduling plan, and make deviation-driven adjustments during the execution process;

[0011] After the task is completed, perform experience accumulation and update the historical experience database.

[0012] Furthermore, for each subtask node in the current task execution logic graph, the computing power scheduling platform predicts the expected duration of the subtask. Based on the resource type, resource preference tags, and input data size of the current subtask, it retrieves at least three historical records most similar to the subtask from the historical experience database. The similarity is judged based on the following criteria: the task type is the same, the resource preference tags are consistent, and the difference in input data size does not exceed ±20%. The weighted average of the actual execution durations of the same subtasks in the historical records is calculated. In the calculation of the weighted average, the weights of each actual execution duration are assigned to 0.5, 0.3, and 0.2 respectively according to their execution time from recent to distant. The weighted average is used as the expected duration of the current subtask.

[0013] Furthermore, for each subtask node in the current task execution logic graph, the computing power scheduling platform predicts the resource requirement vector of the subtask and generates an eight-dimensional resource requirement vector Rv for each subtask node, Rv={CPU core count, memory size, GPU model index, GPU memory size, network bandwidth, storage IOPS, expected start time offset, expected duration}.

[0014] Furthermore, the computing power scheduling platform arranges the resource demand vectors of all subtasks on the time axis according to the expected start time offset based on the sequential order of each subtask node in the task execution logic graph. For subtasks with overlapping times, the computing power resource demand at the same time point is superimposed according to resource type to generate a curve with time as the horizontal axis and the resource demand of each resource type as the vertical axis. This curve is used as the resource demand curve.

[0015] Furthermore, based on the resource demand curve, the computing power scheduling platform generates a scheduling scheme. For the scheduling strategy, the life cycle of the task is divided into multiple consecutive time stages according to the subtask boundaries. Each time stage corresponds to a subtask or a group of concurrently executed subtasks. For each time stage, based on the peak characteristics of the resource demand curve of that time stage, a strategy is selected from the preset scheduling strategy library. The preset scheduling strategy library includes five strategies: memory and IO affinity scheduling strategy, computing power aggregation and exclusive scheduling strategy, network topology-aware scheduling strategy, load balancing round-robin strategy, and resource compression and sharing strategy.

[0016] The computing power scheduling platform compares the resource demand within each 60-second time window on the resource demand curve with the available resources of the nodes within that time window, calculates the matching score for each strategy, and selects the strategy with the highest matching score as the scheduling strategy for that time window.

[0017] Furthermore, based on the resource demand curve, the computing power scheduling platform generates a scheduling scheme. For the elastic resource reservation plan, a resource reservation schedule is formulated according to the resource demand curve. The resource reservation schedule is in seconds and records the computing power resource demand of each resource type that needs to be reserved every second in the next 3 minutes. It includes two rules: advance activation reservation rules and resource smooth release and recycling rules.

[0018] Furthermore, after generating the scheduling scheme, the computing power scheduling platform executes the scheduling according to the scheduling scheme, distributing each subtask to the corresponding target computing node for execution according to its corresponding scheduling strategy and elastic resource reservation plan;

[0019] During the scheduling process, the computing power scheduling platform continuously collects the actual resource usage data of the currently executing subtasks from each computing node at a sampling frequency of once per second. It then compares the actual resource usage data with the predicted values ​​at the corresponding time points on the resource demand curve item by item, calculates the deviation percentage for each resource dimension, and triggers a replanning event when the absolute value of the deviation percentage is greater than the preset deviation threshold.

[0020] When a rescheduling event is triggered, the computing power scheduling platform analyzes the cause of the deviation and compares the actual execution progress of the current subtask with the expected progress. If the actual execution time of the current subtask has exceeded 110% of the expected duration and the actual usage of at least one key resource dimension is greater than or equal to 95% of the predicted value, the deviation is determined to be caused by the computing task being more demanding than expected. If the actual resource usage is less than 80% of the predicted value and the actual usage of at least one key resource dimension is less than 60% of the predicted value, the deviation is determined to be caused by the task's resource requirements being overestimated.

[0021] To correct the deviation caused by the computing task being more demanding than expected, an online resource expansion instruction is sent to the computing power scheduling platform, requesting temporary resource expansion for the computing node where the current subtask is located. The type and quantity of expanded resources are determined based on the current actual resource gap: if the GPU utilization rate is consistently higher than 95% and the video memory usage exceeds 90%, then an additional GPU of the same model is requested, and 50% of the computing load of the subtask is migrated to the new GPU.

[0022] If CPU utilization remains above 90% and memory usage exceeds 85%, then request an additional 4 CPU cores and 8GB of memory.

[0023] If the network bandwidth utilization rate remains above 90%, request a temporary allocation of an independent virtual network channel for this subtask and raise the network priority to the highest level.

[0024] The success of an online resource expansion command depends on whether the computing power scheduling platform currently has enough idle resources. If there are no idle resources, a resource demand curve correction is triggered.

[0025] Based on the current subtask's execution progress, recalculate the revised expected duration of all subsequent subtasks that have not yet started. Use the ratio of the subtask's expected duration to its actual execution duration as the current subtask's execution speed parameter, and use the ratio of each subsequent subtask's expected duration to the current subtask's execution speed parameter as the revised expected duration.

[0026] Based on the proportion by which the actual resource usage of the current subtask exceeds the predicted value, the corresponding resource dimension of the subsequent subtask is proportionally amplified, and the predicted value of the corresponding resource dimension of the subsequent subtask is multiplied by the amplification factor.

[0027] The remaining expected execution time of the current subtask is recalculated as the remaining execution duration, and this remaining execution duration is added to the start time offset of all subsequent subtasks.

[0028] The revised resource demand curve will be re-executed to implement the elastic resource reservation plan. For resources that have been reserved but not yet used, if the revised curve shows a decrease in demand, the excess portion will be released immediately. If the demand increases, an application will be made to the computing power scheduling platform for additional reservations.

[0029] To correct the deviation caused by an overestimation of task resource requirements, an online resource reduction instruction is sent to the computing power scheduling platform, requesting the reclamation of a portion of the resources currently occupied by the subtask. The reclamation rules are as follows:

[0030] If the GPU utilization rate is consistently below 50% and the video memory usage is below 40%, then when there are multiple GPUs, one GPU will be reclaimed and the computing tasks on that GPU will be migrated to the remaining GPUs. Before the migration, it will be checked whether the remaining video memory of the remaining GPUs is sufficient to handle the entire load. If it is insufficient, the GPU will not be reclaimed for the time being.

[0031] If CPU utilization remains below 40%, then if the current allocation exceeds 8 cores, 4 CPU cores will be reclaimed; if the current allocation is between 4 and 8 cores, 2 cores will be reclaimed.

[0032] If the network bandwidth utilization rate remains below 30%, the network priority of the subtask will be reduced to the normal level, and the independent virtual channel it occupies will be reclaimed so that it can share network bandwidth with other tasks.

[0033] Based on the actual resource consumption of the current subtask, reduce the resource requirements of subsequent subtasks:

[0034] Calculate the resource redundancy coefficient of the current subtask, and divide the predicted value of the corresponding resource dimension of the subsequent subtask by the resource redundancy coefficient.

[0035] Adjust the expected duration of subsequent subtasks by multiplying the expected duration of subsequent subtasks by a reduction ratio, where the reduction ratio is the geometric mean of the predicted values ​​at the corresponding time points on the CPU and GPU two-dimensional curves (actual resource usage / resource demand).

[0036] Adjust the expected start time offset of subsequent subtasks. If the current subtask is completed ahead of schedule, the start time offset of subsequent subtasks should be advanced accordingly, with the advance amount equal to the expected remaining time.

[0037] Furthermore, once all subtasks corresponding to a computing power scheduling request have been executed and the execution results have been returned to the computing power demand side, the computing power scheduling platform packages the entire process data of this task from receiving the request to its completion into a scheduling case record and stores the scheduling case record as a historical experience library.

[0038] On the other hand, the present invention also provides an AI-based computing power scheduling device, comprising:

[0039] The graph construction module is used to receive and parse computing power scheduling requests in order to construct a task execution logic graph.

[0040] The predictive analytics module is used to generate resource demand curves based on the task execution logic graph and historical experience database.

[0041] The scheme generation module is used to build a scheme generation model based on the resource demand curve in order to generate scheduling schemes;

[0042] The execution feedback module is used to execute scheduling according to the scheduling scheme and make deviation-driven adjustments during the execution process;

[0043] The data storage module is used to perform experience accumulation and update the historical experience database after the task is completed.

[0044] On the other hand, the present invention also provides a storage medium storing instructions that, when run on a computer, cause the computer to execute the AI-based computing power scheduling method described above.

[0045] The beneficial effects of this invention are as follows: By constructing a task execution logic graph, generating resource demand curves, introducing a deviation-driven dynamic adjustment mechanism, and continuously updating the historical experience database, a closed-loop intelligent scheduling system is formed. This solution can accurately predict resource demands based on the internal structure of the task, dynamically match scheduling strategies with elastic resource reservation plans, and perceive deviations in real time during execution, proactively expanding or shrinking resource capacity. This improves the overall utilization efficiency of heterogeneous computing resources, shortens task completion time, enhances the scheduling system's adaptability to task fluctuations and resource contention, and provides reusable scheduling experience for subsequent tasks, enabling the continuous evolution and optimization of scheduling strategies. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a flowchart of the AI-based computing power scheduling method in this embodiment.

[0048] Figure 2 This is a flowchart of the method for generating the resource demand curve in this embodiment.

[0049] Figure 3 This is a flowchart of the method for generating the scheduling scheme in this embodiment.

[0050] Figure 4 This is a schematic diagram of the AI-based computing power scheduling device in this embodiment. Detailed Implementation

[0051] The AI-based computing power scheduling method, apparatus, and storage medium disclosed in this invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the technical features or combinations of technical features described in the following embodiments should not be considered isolated; they can be combined with each other to achieve better technical effects. In the accompanying drawings of the following embodiments, the same reference numerals appearing in each drawing represent the same features or components, which can be applied to different embodiments. Therefore, once an item is defined in one drawing, it does not need to be further discussed in subsequent drawings.

[0052] It should be noted that the structures, proportions, sizes, etc., illustrated in the accompanying drawings are merely for illustrative purposes and to aid those skilled in the art in understanding and reading the invention. They are not intended to limit the conditions under which the invention can be implemented. Any modifications to the structure, changes in proportions, or adjustments to size, provided they do not affect the effectiveness or purpose of the invention, should fall within the scope of the technical content disclosed in the invention. The scope of the preferred embodiments of the present invention includes other implementations, wherein functions may be performed not in the order stated or discussed, including substantially simultaneously or in reverse order, depending on the functions involved. This should be understood by those skilled in the art to which the embodiments of the present invention pertain.

[0053] Techniques, methods, and apparatus known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and apparatus should be considered part of the specification. In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0054] In the description of the embodiments of this application, " / " means "or", and "and / or" is used to describe the relationship between related objects, indicating that there can be three relationships. For example, "A and / or B" means: A and B exist alone, B exists alone, and A and B exist simultaneously. In the description of the embodiments of this application, "multiple" refers to two or more embodiments.

[0055] Please see Figure 1 As shown, this is the AI-based computing power scheduling method in this embodiment, including:

[0056] Step S1: Receive and parse the computing power scheduling request to construct a task execution logic graph. The computing power scheduling request is received from the computing power demand side through the computing power scheduling platform. It includes computing power resource demand, business application identifier, business type identifier, and task execution logic description file. The task execution logic description file is written in a structured markup language, which is used to define the multiple sub-task units contained in the task to be processed, the execution order between each sub-task unit, the dependency relationship, and the resource preference tags of each sub-task unit. The structured markup language includes XML and JSON formats, etc.

[0057] For a task execution logic description file, such as one used for deep learning model training, it includes the following sub-task units:

[0058] First subtask: Data preprocessing and loading, resource preference tags are high memory bandwidth and high-speed CPU;

[0059] The second subtask: forward and backward computation, with resource preference tags being high-performance GPUs and large-capacity video memory;

[0060] The third subtask: gradient synchronization and parameter update, with the resource preference label being low latency and high bandwidth network;

[0061] The fourth subtask is model validation and saving, with the resource preference tag set to high-speed local storage.

[0062] Specifically, this embodiment is applied to a computing power scheduling platform, which is communicatively connected to both the computing power demand side and the computing power supply side. The computing power supply side includes multiple heterogeneous computing nodes, each of which has computing power resources of at least one resource type. The resource types of the computing power resources include, but are not limited to, the number of CPU cores, the model and memory capacity of the graphics processor, the memory capacity, the network bandwidth, and the disk input / output capability.

[0063] Specifically, in step S1 of this embodiment, after the computing power scheduling platform receives the request, it starts a task parser. The task parser reads the task execution logic description file and converts it into a directed acyclic graph as a task execution logic graph. In the task execution logic graph, each node represents a sub-task unit, and the directed edges between nodes represent the execution order and dependency relationship. The task parser also attaches resource preference tags extracted from the task execution logic description file to each node.

[0064] Specifically, in step S1 of this embodiment, the complex task is broken down into sub-task units with resource preference labels by parsing the task execution logic description file, and a directed acyclic graph is constructed to represent the dependencies and execution order between sub-tasks. This helps the scheduling system to deeply understand the internal structure and resource requirement characteristics of the task, laying a structured foundation for subsequent refined resource prediction and scheduling decisions, and avoiding the resource waste and dependency conflict risks caused by black-box scheduling of tasks as a whole.

[0065] Please continue reading. Figure 1 As shown, the AI-based computing power scheduling method further includes:

[0066] Step S2: Based on the task execution logic graph and historical experience base, generate a resource demand curve. The historical experience base stores various records of historically executed tasks. Each record includes: task type identifier, sub-task sequence, actual execution time of each sub-task, actual amount of various resources used, resource utilization rate, etc.

[0067] Please see Figure 2 As shown, this is a method for generating resource demand curves, including:

[0068] Step S21: Predict the expected duration of the subtask.

[0069] Specifically, in step S21 of this embodiment, for each subtask node in the current task execution logic graph, the computing power scheduling platform predicts the expected duration of the subtask. Based on the resource type, resource preference label, and input data scale of the current subtask, it retrieves at least three historical records most similar to the subtask from the historical experience database. The similarity is judged based on the following criteria: the task type is the same, the resource preference label is consistent, and the difference in input data scale does not exceed ±20%. The weighted average of the actual execution duration of the same subtask in the historical records is calculated. In the calculation of the weighted average, the weights of each actual execution duration are assigned to 0.5, 0.3, and 0.2 respectively according to their execution time from near to far. The weighted average is used as the expected duration of the current subtask, and its unit is seconds.

[0070] Specifically, in step S21 of this embodiment, when analyzing the expected duration, for example, if three historical records are retrieved and arranged in order of execution time from most recent to oldest, with corresponding execution durations of 1800 seconds, 1920 seconds, and 2100 seconds respectively, then the expected duration = 1800 × 0.5 + 1920 × 0.3 + 2100 × 0.2 = 1896 seconds.

[0071] Please continue reading. Figure 2 As shown, the method for generating the resource demand curve further includes:

[0072] Step S22: Predict the resource requirement vector for the subtask.

[0073] Specifically, in step S22 of this embodiment, for each subtask node in the current task execution logic graph, the computing power scheduling platform predicts the resource requirement vector of the subtask and generates an eight-dimensional resource requirement vector Rv for each subtask node. Rv = {CPU core count, memory size, GPU model index, GPU memory size, network bandwidth, storage IOPS, expected start time offset, expected duration}. The units for memory size and GPU memory size are GB, the unit for network bandwidth is Mbps, and the units for expected start time offset and expected duration are s. The references are preset integers, such as 1 for RTX3090, 2 for RTX4050, etc. For the acquisition of data values ​​of each dimension in the resource demand vector, the number of CPU cores, memory size, GPU memory size, network bandwidth, and storage IOPS are obtained by the computing power scheduling platform based on the resource preference tag of the current subtask. The platform retrieves the 75th percentile of the resources used by historical tasks under the same resource preference tag from the historical experience library. The expected start time offset represents the delayed start time of the current subtask relative to the global start time of the task. Its value is determined by the sum of the expected durations of all the preceding subtasks of the subtask in the task execution logic graph.

[0074] Please continue reading. Figure 2 As shown, the method for generating the resource demand curve further includes:

[0075] Step S23: Generate a resource demand curve based on the resource demand vector.

[0076] Specifically, in step S23 of this embodiment, the computing power scheduling platform arranges the resource demand vectors of all subtasks on the time axis according to the expected start time offset based on the sequential order of each subtask node in the task execution logic graph. For subtasks with overlapping times, the computing power resource demand at the same time point is superimposed according to the resource type to generate a curve with time as the horizontal axis and the resource demand of each resource type as the vertical axis. This curve is used as the resource demand curve, which includes at least the CPU demand curve, memory demand curve, GPU memory demand curve, and network bandwidth demand curve.

[0077] Specifically, in step S2 of this embodiment, the expected duration and multi-dimensional resource demand vectors for each subtask are predicted by combining historical execution records and the logical graph of the current task, and then superimposed on the time axis to form a resource demand curve. This allows the scheduling system to anticipate the peak and flat periods of demand for various resources in different time periods before the task starts, thereby guiding the formulation of subsequent scheduling strategies and reservation plans, and significantly improving the accuracy and interpretability of resource demand prediction.

[0078] Please continue reading. Figure 1As shown, the AI-based computing power scheduling method further includes:

[0079] Step S3: Construct a scheme generation model based on the resource demand curve to generate a scheduling scheme, which includes a scheduling strategy and an elastic resource reservation plan.

[0080] Please see Figure 3 As shown, this is a method for generating a scheduling scheme, including:

[0081] Step S31: Generate a scheduling strategy based on the resource demand curve.

[0082] Specifically, in step S31 of this embodiment, based on the resource demand curve, the computing power scheduling platform generates a scheduling scheme. For the scheduling strategy, the lifecycle of a task is divided into multiple consecutive time stages according to the subtask boundaries. Each time stage corresponds to a subtask or a group of concurrently executed subtasks. For each time stage, based on the peak characteristics of that time stage on the resource demand curve, a strategy is selected from a preset scheduling strategy library. The preset scheduling strategy library includes five strategies: memory and I / O affinity scheduling strategy, computing power aggregation and exclusive scheduling strategy, network topology-aware scheduling strategy, load balancing round-robin strategy, and resource compression and sharing strategy.

[0083] For memory and IO affinity scheduling strategy, it is applicable to the time period when memory or storage IOPS in the resource demand curve reaches its peak. It requires scheduling the target subtask to a compute node with local NVMe solid disk and memory bandwidth greater than or equal to 80GB / s, and binding it to the CPU core on the compute node.

[0084] For the computing power aggregation and exclusive scheduling strategy, it is applicable to the time period when GPU computing power or GPU memory reaches its peak. It requires locking all GPU resources required by the target subtask in advance and allocating them to the subtask in an exclusive manner, prohibiting other tasks from preempting them until the subtask ends.

[0085] The network topology-aware scheduling strategy is applicable to the time period when network bandwidth is at its peak. It requires that all computing nodes participating in the subtask are located under the same physical switch, or that the number of communication hops between nodes does not exceed 2 hops.

[0086] The load balancing round-robin strategy is suitable for time periods with flat resource demand and no peak demand. It uses a weighted round-robin method to distribute tasks to multiple computing nodes with the same configuration.

[0087] The resource compression and sharing strategy is applicable to time periods when resource demand is less than 30% of the available resources on a single node. It schedules multiple subtasks of the same type to the same computing node and uses container technology to achieve resource isolation and sharing.

[0088] Specifically, in step S31 of this embodiment, the computing power scheduling platform compares the resource demand within each 60-second time window on the resource demand curve with the available resources of the nodes within that time window, calculates the matching score for each strategy, and selects the strategy with the highest matching score as the scheduling strategy for that time window. The matching score is calculated as follows: for each resource type, the ratio of (available nodes / task demand) is calculated. If the ratio is greater than or equal to 1 and less than 1.2, 3 points are awarded; if the ratio is greater than or equal to 1.2 and less than 2, 2 points are awarded; if the ratio is greater than or equal to 2, 1 point is awarded; if the ratio is less than 1, -10 points are awarded. The scores of all resource types are added together to obtain the total score of the strategy.

[0089] Please continue reading. Figure 3 As shown, the method for generating the scheduling scheme further includes:

[0090] Step S32: Generate a flexible resource reservation plan based on the resource demand curve.

[0091] Specifically, in step S32 of this embodiment, based on the resource demand curve, the computing power scheduling platform generates a scheduling scheme. For the elastic resource reservation plan, a resource reservation schedule is formulated according to the resource demand curve. The resource reservation schedule is in seconds and records the computing power resource demand of each resource type that needs to be reserved every second in the next 3 minutes. It includes two rules: advance activation reservation rules and resource smooth release and reclamation rules.

[0092] For the early activation reservation rule, for any subtask, the scheduler issues a reservation instruction with a preset lead time before its expected start time offset. The preset lead time ranges from 10 seconds to 300 seconds.

[0093] Regarding the resource smooth release and reclamation rules, when a subtask is completed, the resources it occupies are not immediately released in full. Based on the resource demand curve of the next adjacent subtask, it is determined which resources can be retained. If the next subtask's demand for the same resource type is greater than or equal to 50% of the current subtask's demand, then 80% of the current subtask's computing power resources of that resource type are retained for the next subtask's continuous use; otherwise, only 20% is retained as a cache, and the rest is released immediately. The preset lead time can be determined according to the resource demand scale of the subtask. For example, when the GPU memory demand is greater than or equal to 32GB, the preset lead time is 120 seconds; when the GPU memory demand is less than 32GB, the preset lead time is 60 seconds, to ensure that resources are ready in a timely manner and avoid resource idleness caused by premature reservation.

[0094] Specifically, in step S32 of this embodiment, an appropriate scheduling strategy is dynamically selected based on the stage characteristics of the resource demand curve, and an elastic resource reservation plan is formulated. This avoids the limitations of a single scheduling strategy in all scenarios, and can flexibly adjust scheduling behavior according to the peak characteristics of different resources such as memory, GPU, and network. At the same time, through the early activation and smooth release mechanism, resource idleness is reduced while ensuring timely resource readiness, thereby improving the economy and execution stability of resource reservation.

[0095] Please continue reading. Figure 1 As shown, the AI-based computing power scheduling method further includes:

[0096] Step S4: Execute the scheduling according to the scheduling scheme, and make deviation-driven adjustments during the execution process.

[0097] Specifically, in step S4 of this embodiment, after generating the scheduling scheme, the computing power scheduling platform executes the scheduling according to the scheduling scheme, and distributes each subtask to the corresponding target computing node for execution according to its corresponding scheduling strategy and elastic resource reservation plan.

[0098] Specifically, in step S4 of this embodiment, during the scheduling process, the computing power scheduling platform continuously collects actual resource usage data of the currently executing subtasks from each computing node at a sampling frequency of once per second. This actual resource usage data includes: actual CPU utilization, actual memory usage, actual GPU memory usage, and actual network throughput. The platform compares the actual resource usage data with the predicted values ​​at the corresponding time points on the resource demand curve, calculating the percentage deviation for each resource dimension. The percentage deviation is calculated as: (Actual resource usage data - Predicted value at the corresponding time point on the resource demand curve) / Predicted value at the corresponding time point on the resource demand curve × 100%. When the absolute value of the percentage deviation exceeds a preset deviation threshold, a replanning event is triggered. The preset deviation threshold is set differently depending on the resource type. For example, for GPU memory and CPU core count, the preset deviation threshold is set to 20%; for network bandwidth and memory, it is set to 30%; and for storage IOPS, it is set to 40%. The actual CPU utilization is a percentage parameter, the actual memory usage and actual GPU memory usage are in GB, and the actual network throughput is in Mbps.

[0099] Specifically, in step S4 of this embodiment, when a replanning event is triggered, the computing power scheduling platform analyzes the cause of the deviation, compares the actual execution progress of the current subtask with the expected progress, and if the actual execution time of the current subtask has exceeded 110% of the expected duration and the actual usage of at least one key resource dimension is greater than or equal to 95% of the predicted value, then the deviation is determined to be caused by the computing task being more demanding than expected; if the actual resource usage is less than 80% of the predicted value and the actual usage of at least one key resource dimension is less than 60% of the predicted value, then the deviation is determined to be caused by the task resource requirements being overestimated. The key resource dimensions include GPU utilization, CPU utilization, and network bandwidth.

[0100] Specifically, in step S4 of this embodiment, in order to correct the deviation caused by the computing task being more demanding than expected, an online resource expansion instruction is sent to the computing power scheduling platform, requesting that resources be temporarily added to the computing node where the current subtask is located. The type and quantity of expanded resources are determined according to the current actual resource gap: if the GPU utilization rate is consistently higher than 95% and the video memory usage exceeds 90%, then an additional GPU of the same model is requested, and 50% of the computing load of the subtask is migrated to the new GPU.

[0101] If CPU utilization remains above 90% and memory usage exceeds 85%, then request an additional 4 CPU cores and 8GB of memory.

[0102] If the network bandwidth utilization rate remains above 90%, request a temporary allocation of an independent virtual network channel for this subtask and raise the network priority to the highest level.

[0103] The success of an online resource expansion command depends on whether the computing power scheduling platform currently has enough idle resources. If there are no idle resources, a resource demand curve correction is triggered.

[0104] Based on the current subtask's execution progress, recalculate the corrected expected duration of all subsequent subtasks that have not yet started. Use the ratio of the expected duration of the subtask to the actual execution duration as the execution speed parameter of the current subtask. Use the ratio of the expected duration of each subsequent subtask to the execution speed parameter of the current subtask as the corrected expected duration. The corrected expected duration is rounded up to an integer multiple of 10 seconds.

[0105] Based on the proportion by which the actual resource usage of the current subtask exceeds the predicted value, the corresponding resource dimension of the subsequent subtask is proportionally amplified. The predicted value of the corresponding resource dimension of the subsequent subtask is multiplied by an amplification factor. The amplification factor is the maximum value among all key resource dimensions (the predicted value at the corresponding time point on the actual resource usage / resource demand curve), and the value of the amplification factor is limited to between 1.1 and 2. If the amplification factor exceeds the boundary value, it is set to the corresponding boundary value.

[0106] The remaining expected execution time of the current subtask is recalculated as the remaining execution duration, where the remaining execution duration = (expected duration - actual execution duration) × (actual execution duration / expected duration), and this remaining execution duration is added to the start time offset of all subsequent subtasks.

[0107] The revised resource demand curve will be used to re-execute the elastic resource reservation plan. For resources that have been reserved but not yet used, if the revised curve shows a decrease in demand, the excess portion will be released immediately. If the demand increases, a supplementary reservation will be requested from the computing power scheduling platform immediately.

[0108] Specifically, in step S4 of this embodiment, to correct the deviation caused by an overestimation of task resource requirements, an online resource reduction instruction is sent to the computing power scheduling platform, requesting the reclamation of a portion of the resources currently occupied by the subtask. The reclamation rules are as follows:

[0109] If the GPU utilization rate is consistently below 50% and the video memory usage is below 40%, then when there are multiple GPUs, one GPU will be reclaimed and the computing tasks on that GPU will be migrated to the remaining GPUs. Before the migration, it will be checked whether the remaining video memory of the remaining GPUs is sufficient to handle the entire load. If it is insufficient, the GPU will not be reclaimed for the time being.

[0110] If the CPU utilization rate remains below 40%, then if the current allocation exceeds 8 cores, 4 CPU cores will be reclaimed, or if the current allocation is between 4 and 8 cores, 2 cores will be reclaimed.

[0111] If the network bandwidth utilization rate remains below 30%, the network priority of the subtask will be reduced to the normal level, and the independent virtual channel it occupies will be reclaimed so that it can share network bandwidth with other tasks.

[0112] Based on the actual resource consumption of the current subtask, reduce the resource requirements of subsequent subtasks:

[0113] Calculate the resource redundancy coefficient of the current subtask, and divide the predicted value of the corresponding resource dimension of the subsequent subtask by the resource redundancy coefficient. The resource redundancy coefficient is the maximum value of (predicted value at the corresponding time point on the resource demand curve / actual resource usage) among all key resource dimensions, and the resource redundancy coefficient is limited to between 1.1 and 3. If the resource redundancy coefficient exceeds the boundary value, it is set to the corresponding boundary value.

[0114] Adjust the expected duration of subsequent subtasks by multiplying the expected duration of subsequent subtasks by a reduction ratio. The reduction ratio is the geometric mean of the predicted values ​​at the corresponding time points on the actual resource usage / resource demand curves in both CPU and GPU dimensions. The minimum value of the reduction ratio is 0.3.

[0115] Adjust the expected start time offset of subsequent subtasks. If the current subtask is completed ahead of schedule, the start time offset of subsequent subtasks should be advanced accordingly, and the advance amount is equal to the expected remaining time.

[0116] Specifically, in step S4 of this embodiment, the online resource shrinkage instruction should ensure that the current subtask will not experience new execution delays due to resource reduction. Before performing the recycling, the resource utilization rate after recycling will be predicted: if the predicted utilization rate after shrinkage will exceed 85%, then the recycling operation will be abandoned.

[0117] For cases that do not meet any of the above rules, such as slightly longer actual execution time but lower resource utilization or higher actual resource usage but normal execution time, they are classified as mixed deviations and handled according to the most conservative strategy to ensure system stability. The most conservative strategy is to adopt the scheme with the greater increase in resource demand in both correction directions.

[0118] Specifically, in step S4 of this embodiment, actual resource usage data is sampled at high frequency and compared dimension-by-dimensionally with the resource demand curve. When the deviation exceeds a threshold, a replanning event is triggered, and expansion or contraction correction is performed based on the cause of the deviation. This enables the scheduling system to have real-time perception and dynamic response capabilities, allowing it to proactively adjust resource allocation when task execution anomalies occur or demands change, avoiding task delays due to insufficient resources or waste due to excess resources, and significantly enhancing the robustness and adaptability of scheduling.

[0119] Please continue reading. Figure 1 As shown, the AI-based computing power scheduling method further includes:

[0120] Step S5: After the task is completed, perform experience accumulation and update the historical experience database.

[0121] Specifically, in step S5 of this embodiment, after all subtasks corresponding to a computing power scheduling request have been executed and the execution results have been returned to the computing power demand side, the computing power scheduling platform packages the entire process data of this task from receiving the request to the completion of execution into a scheduling case record and stores the scheduling case record as a historical experience library. The scheduling case record includes: task identifier code, task execution logic graph, predicted multi-dimensional resource demand curve, actual collected resource usage curve, time points of all deviation triggering events, deviation reasons, replanning actions taken, and the final total task execution time and total resource consumption cost.

[0122] Specifically, in step S5 of this embodiment, the predicted data, actual data, deviation events, and adjustment actions during the complete task execution process are packaged into scheduling case records and stored in the historical experience database. This provides real and traceable experience for resource prediction and scheduling decisions for subsequent tasks, enabling the scheduling system to continuously learn from historical successes and failures, gradually optimize the accuracy of the resource demand model and the rationality of strategy selection, and form a data-driven closed-loop evolution mechanism.

[0123] Please see Figure 4 As shown, this is the AI-based computing power scheduling device in this embodiment, including:

[0124] The graph construction module is used to receive and parse computing power scheduling requests in order to construct a task execution logic graph.

[0125] The predictive analytics module is used to generate resource demand curves based on the task execution logic graph and historical experience database.

[0126] The scheme generation module is used to build a scheme generation model based on the resource demand curve in order to generate scheduling schemes;

[0127] The execution feedback module is used to execute scheduling according to the scheduling scheme and make deviation-driven adjustments during the execution process;

[0128] The data storage module is used to perform experience accumulation and update the historical experience database after the task is completed.

[0129] This application also provides a computer-readable storage medium storing instructions that, when run on a computer, cause the computer to execute the AI-based computing power scheduling method as described in the above method embodiments.

[0130] It will be understood by those skilled in the art that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable programs, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically contain computer-readable programs, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0131] In the above description, the disclosure of this invention is not intended to limit itself to these aspects. Rather, within the scope of the objectives of this disclosure, components can be selectively and operationally combined in any number. Furthermore, terms such as “comprising,” “encompassing,” and “having” should be interpreted by default as inclusive or open-ended, rather than exclusive or closed, unless explicitly defined as such. All technical, scientific, or other terms are to be understood by those skilled in the art, unless defined as such. Public terms found in dictionaries should not be interpreted in the context of the relevant technical documents in an overly idealistic or impractical manner, unless explicitly defined as such in this disclosure. Any modifications or alterations made by those skilled in the art based on the foregoing disclosure are within the scope of the claims.

Claims

1. An AI-based computing power scheduling method, characterized in that, include: Receive and parse computing power scheduling requests to construct a task execution logic graph; Based on the task execution logic graph and historical experience database, a resource requirement curve is generated. A scheme generation model is constructed based on the resource demand curve to generate scheduling schemes; Execute scheduling according to the scheduling plan, and make deviation-driven adjustments during the execution process; After the task is completed, perform experience accumulation and update the historical experience database.

2. The AI-based computing power scheduling method according to claim 1, characterized in that, For each subtask node in the current task execution logic graph, the computing power scheduling platform predicts the expected duration of the subtask. Based on the resource type, resource preference tags, and input data size of the current subtask, it retrieves at least three historical records most similar to the subtask from the historical experience database. The similarity is judged based on the following criteria: the task type is the same, the resource preference tags are consistent, and the difference in input data size does not exceed ±20%. The weighted average of the actual execution durations of the same subtasks in the historical records is calculated. In the calculation of the weighted average, the weights of each actual execution duration are assigned to 0.5, 0.3, and 0.2 respectively according to their execution time from most recent to oldest. The weighted average is used as the expected duration of the current subtask.

3. The AI-based computing power scheduling method according to claim 2, characterized in that, For each subtask node in the current task execution logic graph, the computing power scheduling platform predicts the resource requirement vector of the subtask and generates an eight-dimensional resource requirement vector Rv for each subtask node, Rv={number of CPU cores, memory size, GPU model index, GPU memory size, network bandwidth, storage IOPS, expected start time offset, expected duration}.

4. The AI-based computing power scheduling method according to claim 3, characterized in that, The computing power scheduling platform arranges the resource demand vectors of all subtasks on the time axis according to the expected start time offset based on the sequential order of each subtask node in the task execution logic graph. For subtasks with overlapping times, the computing power resource demand at the same time point is superimposed according to resource type to generate a curve with time as the horizontal axis and the resource demand of each resource type as the vertical axis. This curve is used as the resource demand curve.

5. The AI-based computing power scheduling method according to claim 4, characterized in that, Based on the resource demand curve, the computing power scheduling platform generates a scheduling scheme. For the scheduling strategy, the life cycle of the task is divided into multiple consecutive time stages according to the subtask boundary. Each time stage corresponds to a subtask or a group of concurrently executed subtasks. For each time stage, based on the peak characteristics of the resource demand curve of that time stage, a strategy is selected from the preset scheduling strategy library. The preset scheduling strategy library includes five strategies: memory and IO affinity scheduling strategy, computing power aggregation and exclusive scheduling strategy, network topology-aware scheduling strategy, load balancing round-robin strategy, and resource compression and sharing strategy. The computing power scheduling platform compares the resource demand within each 60-second time window on the resource demand curve with the available resources of the nodes within that time window, calculates the matching score for each strategy, and selects the strategy with the highest matching score as the scheduling strategy for that time window.

6. The AI-based computing power scheduling method according to claim 5, characterized in that, Based on the resource demand curve, the computing power scheduling platform generates a scheduling scheme. For the elastic resource reservation plan, a resource reservation schedule is formulated according to the resource demand curve. The resource reservation schedule is in seconds and records the computing power resource demand of each resource type that needs to be reserved every second in the next 3 minutes. It includes two rules: advance activation reservation rules and resource smooth release and recycling rules.

7. The AI-based computing power scheduling method according to claim 6, characterized in that, After generating the scheduling scheme, the computing power scheduling platform executes the scheduling according to the scheduling scheme, distributing each subtask to the corresponding target computing node for execution according to its corresponding scheduling strategy and elastic resource reservation plan; During the scheduling process, the computing power scheduling platform continuously collects the actual resource usage data of the currently executing subtasks from each computing node at a sampling frequency of once per second. It then compares the actual resource usage data with the predicted values ​​at the corresponding time points on the resource demand curve item by item, calculates the deviation percentage for each resource dimension, and triggers a replanning event when the absolute value of the deviation percentage is greater than the preset deviation threshold. When a replanning event is triggered, the computing power scheduling platform analyzes the cause of the deviation and compares the actual execution progress of the current subtask with the expected progress. If the actual execution time of the current subtask has exceeded 110% of the expected duration and the actual usage of at least one key resource dimension is greater than or equal to 95% of the predicted value, then the deviation is determined to be caused by the computing task being more demanding than expected. If the actual resource usage is less than 80% of the predicted value and the actual usage of at least one key resource dimension is less than 60% of the predicted value, then the deviation is judged to be due to the task resource requirements being overestimated. To correct the deviation caused by the computing task being more demanding than expected, an online resource expansion instruction is sent to the computing power scheduling platform, requesting temporary resource expansion for the computing node where the current subtask is located. The type and quantity of expanded resources are determined based on the current actual resource gap: if the GPU utilization rate is consistently higher than 95% and the video memory usage exceeds 90%, then an additional GPU of the same model is requested, and 50% of the computing load of the subtask is migrated to the new GPU. If CPU utilization remains above 90% and memory usage exceeds 85%, then request an additional 4 CPU cores and 8GB of memory. If the network bandwidth utilization rate remains above 90%, request a temporary allocation of an independent virtual network channel for this subtask and raise the network priority to the highest level. The success of an online resource expansion command depends on whether the computing power scheduling platform currently has enough idle resources. If there are no idle resources, a resource demand curve correction is triggered. Based on the current subtask's execution progress, recalculate the revised expected duration of all subsequent subtasks that have not yet started. Use the ratio of the subtask's expected duration to its actual execution duration as the current subtask's execution speed parameter, and use the ratio of each subsequent subtask's expected duration to the current subtask's execution speed parameter as the revised expected duration. Based on the proportion by which the actual resource usage of the current subtask exceeds the predicted value, the corresponding resource dimension of the subsequent subtask is proportionally amplified, and the predicted value of the corresponding resource dimension of the subsequent subtask is multiplied by the amplification factor. The remaining expected execution time of the current subtask is recalculated as the remaining execution duration, and this remaining execution duration is added to the start time offset of all subsequent subtasks. The revised resource demand curve will be re-executed to implement the elastic resource reservation plan. For resources that have been reserved but not yet used, if the revised curve shows a decrease in demand, the excess portion will be released immediately. If the demand increases, an application will be made to the computing power scheduling platform for additional reservations. To correct the deviation caused by an overestimation of task resource requirements, an online resource reduction instruction is sent to the computing power scheduling platform, requesting the reclamation of a portion of the resources currently occupied by the subtask. The reclamation rules are as follows: If the GPU utilization rate is consistently below 50% and the video memory usage is below 40%, then when there are multiple GPUs, one GPU will be reclaimed and the computing tasks on that GPU will be migrated to the remaining GPUs. Before the migration, it will be checked whether the remaining video memory of the remaining GPUs is sufficient to handle the entire load. If it is insufficient, the GPU will not be reclaimed for the time being. If CPU utilization remains below 40%, then if the current allocation exceeds 8 cores, 4 CPU cores will be reclaimed; if the current allocation is between 4 and 8 cores, 2 cores will be reclaimed. If the network bandwidth utilization rate remains below 30%, the network priority of the subtask will be reduced to the normal level, and the independent virtual channel it occupies will be reclaimed so that it can share network bandwidth with other tasks. Based on the actual resource consumption of the current subtask, reduce the resource requirements of subsequent subtasks: Calculate the resource redundancy coefficient of the current subtask, and divide the predicted value of the corresponding resource dimension of the subsequent subtask by the resource redundancy coefficient. Adjust the expected duration of subsequent subtasks by multiplying the expected duration of subsequent subtasks by a reduction ratio, where the reduction ratio is the geometric mean of the predicted values ​​at the corresponding time points on the CPU and GPU two-dimensional curves (actual resource usage / resource demand). Adjust the expected start time offset of subsequent subtasks. If the current subtask is completed ahead of schedule, the start time offset of subsequent subtasks should be advanced accordingly, with the advance amount equal to the expected remaining time.

8. The AI-based computing power scheduling method according to claim 7, characterized in that, Once all subtasks corresponding to a computing power scheduling request have been executed and the execution results have been returned to the computing power demand side, the computing power scheduling platform packages the entire process data of this task from receiving the request to its completion into a scheduling case record and stores the scheduling case record as a historical experience library.

9. An AI-based computing power scheduling device, applied to the AI-based computing power scheduling method as described in any one of claims 1-8, characterized in that, include: The graph construction module is used to receive and parse computing power scheduling requests in order to construct a task execution logic graph. The predictive analytics module is used to generate resource demand curves based on the task execution logic graph and historical experience database. The scheme generation module is used to build a scheme generation model based on the resource demand curve in order to generate scheduling schemes; The execution feedback module is used to execute scheduling according to the scheduling scheme and make deviation-driven adjustments during the execution process; The data storage module is used to perform experience accumulation and update the historical experience database after the task is completed.

10. A storage medium, characterized in that, The system stores instructions that, when executed on a computer, cause the computer to perform the AI-based computing power scheduling method as described in any one of claims 1-8.

Images