Optimization method and device for dynamic migration and resource rearrangement of cloud native AI task, equipment and medium

Abstract

The invention discloses a cloud native AI task dynamic migration and resource rearrangement optimization method and device, equipment and a medium, and relates to the technical field of communication, the method comprises the following steps: in response to a resource scheduling request for a target AI task, obtaining a resource demand and a hardware topology constraint condition of the target AI task, and executing an initial resource allocation attempt for the target AI task; if the initial resource allocation attempt fails, constructing a global scheduling task set including the target AI task and the existing migratable tasks in the cluster; all tasks in the global scheduling task set serve as objects to be scheduled, a resource rearrangement model is established and solved, and a target resource distribution scheme is obtained; and comparing the target resource distribution scheme with the current resource distribution state, generating a resource adjustment operation, and migrating the cluster resource state from the current distribution to the target distribution. According to the method, the deployment success rate of the complex AI task in a high-demand or fragmented resource environment and the overall resource utilization rate of the cluster are effectively improved.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to an optimization method, apparatus, device and medium for dynamic migration and resource rearrangement of cloud-native AI tasks. Background Technology

[0002] In existing cloud-native environments and cluster management systems, container orchestration platforms typically use Pods as the smallest deployable unit for resource scheduling and management. Current mainstream scheduling mechanisms employ a "static fill" model, where the scheduler, upon receiving a Pod creation request, filters and scores nodes in the cluster based solely on their current resource availability. If a free node exists that meets the resource requirements, the Pod is bound to that node. Once bound, the Pod-node correspondence remains fixed; the scheduler will not actively intervene or adjust the allocated resource locations until the Pod is evicted due to failure or human intervention.

[0003] However, the aforementioned static scheduling methods exhibit significant limitations when facing artificial intelligence scenarios, especially large language model training and inference. Because such tasks impose strict constraints on hardware topology—for example, requiring multiple GPUs to be located on the same PCIe switch to ensure communication bandwidth, or prohibiting computational tasks from executing across nodes—traditional schedulers rely solely on instantaneous resource snapshots for decision-making. This makes them unable to proactively migrate or reorganize existing tasks to meet these topological constraints when resources are abundant but fragmented. Consequently, resource utilization efficiency is low, task deployment failure rates are high, and they cannot meet the complex and ever-changing resource orchestration needs of AI tasks in cloud-native environments. Summary of the Invention

[0004] This application provides an optimization method, apparatus, device, and medium for dynamic migration and resource rearrangement of cloud-native AI tasks, which can solve the deployment failure problem of large AI models in cloud-native environments caused by resource fragmentation and hardware topology constraints, and achieve efficient resource utilization through dynamic migration and intelligent rearrangement.

[0005] To achieve the above objectives, this application adopts the following technical solution: Firstly, this application provides an optimization method for dynamic migration and resource rearrangement of cloud-native AI tasks, the method comprising: In response to a resource scheduling request for a target AI task, the resource requirements and hardware topology constraints of the target AI task are obtained. Based on the current resource distribution status of the cluster and the resource requirements and hardware topology constraints of the target AI task, an initial resource allocation attempt is performed for the target AI task. If the initial resource allocation attempt fails, a resource rearrangement process is triggered; the resource rearrangement process includes: Construct a global scheduling task set that includes the target AI task and existing transferable tasks in the cluster; Using all tasks in the global scheduling task set as the scheduling objects, a resource rearrangement model is established with the optimization objective of minimizing the number of task migrations. The resource rearrangement model includes node-level constraints and device-level constraints based on the cluster hardware topology. Solve the resource rearrangement model to obtain the target resource distribution scheme; The target resource distribution scheme is compared with the current resource distribution status, and a resource adjustment operation is generated to migrate the cluster resource status from the current distribution to the target distribution.

[0006] In some possible implementations, the initial resource allocation attempt for the target AI task includes: Based on the resource requirements and hardware topology constraints, a set of idle computing devices that meet the conditions is searched in the cluster. If a set of idle computing devices that meet the conditions is found, the target AI task is assigned to the set of idle computing devices that meet the conditions. If no set of idle computing devices that meet the conditions is found, the initial resource allocation attempt is determined to have failed.

[0007] In some possible implementations, the hardware topology constraints include requiring computing devices to be located under the same physical switching unit; if it is not possible to satisfy the requirement that computing devices be located under the same physical switching unit, the constraint degenerates into requiring computing devices to be located within the same computing node.

[0008] In some possible implementations, the method further includes: If the resource rearrangement model has no solution, then based on the historical load data of existing tasks of the same type as the target AI task in the global scheduling task set, the parallel execution configuration of the existing tasks is adjusted; based on the adjusted parallel execution configuration, the resource requirements of the existing tasks in the global scheduling task set are updated; and the resource rearrangement model is re-established using the updated global scheduling task set.

[0009] In some possible implementations, comparing the target resource distribution scheme with the current resource distribution state to generate a resource adjustment operation includes: The target resource distribution scheme is compared with the current resource distribution status to obtain a first comparison result; based on the first comparison result, new task instances to be created, old task instances to be deleted, and task instances to be migrated are determined; the resource adjustment operation is executed serially in the order of first deleting and creating task instances, and then migrating task instances.

[0010] In some possible implementations, when the computing device assigned to the task instance to be migrated needs to change its physical connection topology within the same computing node, the execution of task instance migration includes: Delete the current running instance of the task instance that needs to be migrated; determine the new physical connection order of computing devices for the task instance that needs to be migrated according to the target resource distribution scheme; create and run a new instance of the task instance that needs to be migrated according to the new physical connection order of computing devices.

[0011] In some possible implementations, the resource requirements and hardware topology constraints include the number of computing devices required for the target AI task and the required interconnect topology between the computing devices.

[0012] Secondly, this application provides an optimization device for dynamic migration and resource rearrangement of cloud-native AI tasks, the device comprising: The acquisition module is used to acquire the resource requirements and hardware topology constraints of the target AI task in response to a resource scheduling request for the target AI task. The resource allocation module is used to perform an initial resource allocation attempt for the target AI task based on the current resource distribution status of the cluster and the resource requirements and hardware topology constraints of the target AI task. The resource rearrangement module is used to construct a global scheduling task set, including the target AI task and existing migrateable tasks in the cluster, if the initial resource allocation attempt fails; using all tasks in the global scheduling task set as the scheduling objects, a resource rearrangement model is established with the optimization objective of minimizing the number of task migrations. The resource rearrangement model includes node-level constraints and device-level constraints based on the cluster hardware topology; the resource rearrangement model is solved to obtain the target resource distribution scheme. The resource adjustment module is used to compare the target resource distribution scheme with the current resource distribution status, generate a resource adjustment operation, and migrate the cluster resource status from the current distribution to the target distribution.

[0013] Thirdly, this application provides a computing device, including a memory and a processor; The memory stores one or more computer programs, the one or more computer programs including instructions; when the instructions are executed by the processor, the computing device performs the method as described in any one of the first aspects.

[0014] Fourthly, this application provides a computer-readable storage medium for storing a computer program for performing the method as described in any one of the first aspects.

[0015] Fifthly, this application provides a computer program product comprising one or more computer instructions, wherein when the computer instructions are executed by a computer, the computer performs the method as described in any one of the first aspects.

[0016] As can be seen from the above technical solution, this application has at least the following beneficial effects: In this application, in response to a resource scheduling request for a target AI task, the resource requirements and hardware topology constraints of the target AI task are obtained. Based on the current resource distribution state of the cluster and the resource requirements and hardware topology constraints of the target AI task, an initial resource allocation attempt is performed for the target AI task. If the initial resource allocation attempt fails, a global scheduling task set is constructed, including the target AI task and existing migrateable tasks in the cluster. Using all tasks in the global scheduling task set as the scheduling objects, a resource rearrangement model is established with the optimization objective of minimizing the number of task migrations. The resource rearrangement model is solved to obtain the target resource distribution scheme. The target resource distribution scheme is compared with the current resource distribution state, and a resource adjustment operation is generated to migrate the cluster resource state from the current distribution to the target distribution. In the prior art, traditional static resource schedulers often fail to deploy large AI model tasks that require specific hardware topologies (such as the same PCIe switching unit) due to resource fragmentation, and cannot actively optimize resource configuration to meet such constraints. As can be seen, by introducing a dynamic reordering mechanism that includes topology awareness and consideration of transferable tasks, this application can intelligently readjust the existing task layout when the total amount of resources is sufficient but the distribution does not meet specific constraints, thereby effectively improving the deployment success rate of complex AI tasks and the overall resource utilization of the cluster in high-demand or fragmented resource environments.

[0017] It should be understood that the descriptions of technical features, technical solutions, beneficial effects, or similar language in this application do not imply that all features and advantages can be achieved in any single embodiment. Rather, it is understood that the description of a feature or beneficial effect means that a specific technical feature, technical solution, or beneficial effect is included in at least one embodiment. Therefore, the descriptions of technical features, technical solutions, or beneficial effects in this specification do not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions, and beneficial effects described in this embodiment can be combined in any suitable manner. Those skilled in the art will understand that embodiments can be implemented without one or more specific technical features, technical solutions, or beneficial effects of a particular embodiment. In other embodiments, additional technical features and beneficial effects may be identified in specific embodiments that do not embody all embodiments. Attached Figure Description

[0018] Figure 1 A schematic diagram illustrating an application scenario provided in an embodiment of this application; Figure 2 This application provides an architectural diagram of an intelligent orchestration and rearrangement system. Figure 3 A flowchart illustrating an optimization method for dynamic migration and resource rearrangement of cloud-native AI tasks provided in this application embodiment; Figure 4 A schematic diagram of an optimization device for dynamic migration and resource rearrangement of cloud-native AI tasks; Figure 5 This is a schematic diagram of a computing device provided in an embodiment of this application. Detailed Implementation

[0019] The terms "first," "second," and "third," etc., used in this application specification and accompanying drawings are used to distinguish different objects, not to limit a specific order.

[0020] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0021] To ensure clarity and conciseness in the description of the following embodiments, a brief introduction to the related technologies is given first: Currently, in cloud-native and containerized environments, resource schedulers generally adopt a static fill-in mode. In this mode, when the scheduler receives a Pod creation request, it only matches and scores the remaining resources (such as CPU, memory, and GPU quantity) of each node in the cluster at the current moment, and binds the Pod to the idle node that first meets its basic resource requirements. Once the binding is completed, the correspondence between the Pod and the node is fixed, and the scheduler will not actively intervene unless the Pod is evicted due to failure or manually deleted.

[0022] With the increasing prevalence of complex AI workloads such as large language models, these tasks place stringent requirements on the topology of underlying hardware resources. For example, multiple GPUs must be located in the same PCIe switching unit to ensure high-speed communication bandwidth, or tasks must absolutely not run across physical nodes. Existing static fill-in scheduling strategies, when faced with such requests with strict topology constraints, often fail to find contiguous resource blocks that meet the conditions, even when the overall cluster resources are sufficient, because idle resources are scattered across different switching units or nodes. This leads to task deployment failures and prevents the full utilization of the cluster's computing power.

[0023] In view of this, this application provides an optimization method for dynamic migration and resource rearrangement of cloud-native AI tasks. This method introduces an intelligent rearrangement stage after a conventional initial scheduling failure. This stage constructs a global optimization model that includes new requests and existing migrateable tasks, aiming to minimize migration costs. It solves for the optimal resource distribution scheme that satisfies the hardware topology constraints of all tasks (especially new requests), and adjusts the cluster state from the current distribution to the target distribution through orderly resource adjustment operations (such as task migration, deletion, and reconstruction). This effectively solves the problem of resource fragmentation causing the deployment failure of large AI models.

[0024] To make the technical solution of this application clearer and easier to understand, the application scenarios of the technical solution of this application are described below with reference to the accompanying drawings. Figure 1 As shown in the figure, this figure is a schematic diagram of an application scenario provided by an embodiment of this application.

[0025] In this application scenario, a cloud-native AI cluster consists of multiple physical nodes, each containing multiple GPU computing devices interconnected via PCIe switches. A user submits a large language model training task through the cluster management platform. This task requires exclusive access to eight GPUs, and these GPUs must be located within the same PCIe switch unit to ensure high-speed communication bandwidth. However, the current cluster's GPU resources are fragmented: while the total number of idle GPUs is sufficient, they are scattered across multiple physical nodes and switch units, with no single node or switch unit containing a contiguous set of eight idle GPUs. In this situation, a traditional static scheduler would directly reject the task because it cannot find a contiguous block of resources satisfying the topology constraints, resulting in valuable computing resources being idle while critical tasks cannot run. This situation frequently occurs in production environments, especially when online services and offline training tasks are deployed in a mixed manner, severely impacting the utilization efficiency of cluster resources and the delivery speed of AI services.

[0026] Therefore, the technical solution of this application is introduced into the scheduling system of the cluster. When traditional scheduling fails, the scheduler initiates a dynamic reordering mechanism. Using a linear programming algorithm, it globally analyzes the distribution of existing portable tasks (such as low-priority inference services) and calculates an optimized solution that reorganizes fragmented resources to meet the constraints of the new task. This solution may involve migrating some existing tasks from the current node to other nodes, thereby freeing up the necessary contiguous resource blocks for the target large model task. It is evident that this solution, in scenarios where the total amount of resources is sufficient but the distribution does not meet specific hardware topology constraints, can proactively and intelligently reorder resources to transform previously undeployable tasks into deployable ones, significantly improving the deployment success rate of complex AI tasks in fragmented resource environments and the overall resource utilization efficiency of the cluster.

[0027] As one specific implementation, the optimization method provided in this application embodiment can be executed by an independent intelligent orchestration and rearrangement system. For example... Figure 2 The diagram illustrates the architecture of an intelligent orchestration and rearrangement system. Figure 2 As shown, the system mainly includes the following three core components, which work together to complete a closed loop from resource awareness to decision execution: The scheduler, as the core decision-making unit of the system, is responsible for receiving resource scheduling requests and executing core scheduling and rearrangement algorithms. Its specific functions include: selecting the appropriate initial deployment algorithm based on the type of task request (e.g., large or small model); triggering and executing resource rearrangement algorithms when initial deployment fails, such as building and solving an optimization model; and generating a specific resource adjustment operation sequence based on the solved target resource distribution scheme.

[0028] The Cluster Resource Manager (102) serves as the system's resource map and status maintenance center. It is responsible for collecting and maintaining the cluster's global real-time status. Its specific functions include: initializing and continuously updating the cluster's hardware topology, such as recording the connection and dependency relationships of various compute nodes, PCIe switching units, and GPU devices; maintaining the resource occupancy map of all current Pods, mapping each running Pod instance to its specific physical resources (such as GPU device IDs); and providing the scheduler with an accurate and consistent view of cluster resources for resource lookup and allocation decisions.

[0029] Workers (103) act as execution agents distributed across each compute node. They are responsible for executing specific resource operation commands and reporting some status information. Their functions include: processing northbound operation commands from the scheduler, such as creating (starting) or deleting (releasing) specific Pods on the local node based on the command; retrieving information such as the model image and configuration parameters corresponding to the task from external configuration storage (such as a MySQL database) to instantiate the Pod; and reporting resource status changes of the node, such as GPU usage or release events, to the cluster resource manager to ensure timely updates to the global resource view.

[0030] The three components mentioned above communicate and collaborate through defined interfaces. When a new AI task request arrives, the process can be summarized as follows: Workers receive the request and forward it to the Scheduler; the Scheduler queries the ClusterResource Manager for the current detailed resource topology and occupancy status; based on this information, the Scheduler makes scheduling or reordering decisions and issues the final operation instructions (such as "create Pod P on node A") to the corresponding Worker for execution; the Worker executes the operation and feeds back the result status to the ClusterResourceManager for global status updates. Through this modular and collaborative architecture, the system can effectively combine resource awareness, intelligent decision-making, and precise execution to achieve complex, topology-constrained dynamic resource orchestration.

[0031] To make the technical solution of this application clearer and easier to understand, the following describes an optimization method for dynamic migration and resource rearrangement of cloud-native AI tasks provided by an embodiment of this application, in conjunction with the above application scenarios. Figure 3 As shown in the figure, this is a flowchart of an optimization method for dynamic migration and resource rearrangement of cloud-native AI tasks provided in an embodiment of this application.

[0032] The optimization method for dynamic migration and resource rearrangement of cloud-native AI tasks provided in this application embodiment can be executed by one or more processing devices. The processing device can be a dedicated server, a high-performance workstation located in the cloud-native cluster management plane, or any general-purpose computing device running container orchestration platform control components. The method includes: S201. In response to a resource scheduling request for the target AI task, obtain the resource requirements and hardware topology constraints of the target AI task.

[0033] This step is the initialization and parameter input phase of the scheduling process. When a resource scheduling request for a target AI task is received, the system first parses and extracts its scheduling parameters: Resource requirements specify the total number of graphics processing units (GPUs) required for the task. This parameter directly determines the task's basic computational resource requirements, denoted as . .

[0034] Hardware topology constraints clearly define the performance requirements of the task on the GPU's physical interconnect topology, which is crucial in determining its runtime efficiency. These constraints are particularly stringent for large model tasks. Optimal constraint (same physical switching unit): All GPUs required for the task must be located under the same PCIe Switch to ensure the highest interconnect bandwidth and the lowest communication latency.

[0035] Degradation constraint (same compute node): If the cluster cannot satisfy the same switch constraint, the acceptable performance degradation solution is to require all GPUs to be located in at least the same compute node.

[0036] Small model tasks typically do not have this special constraint.

[0037] Formalize user requests as input parameters for the scheduling system Topological preferences provide a precise data foundation for subsequent mathematical modeling. In particular, determining whether the target AI task is a large-scale model task determines what kind of topological constraints it will be subject to.

[0038] S202. Based on the current resource distribution status of the cluster and the resource requirements and hardware topology constraints of the target AI task, perform an initial resource allocation attempt for the target AI task.

[0039] Based on resource requirements and hardware topology constraints, a set of available computing devices that meet the conditions is searched within the cluster. If a set of available computing devices that meet the conditions is found, the target AI task is assigned to that set of available computing devices. If no set of available computing devices that meet the conditions is found, the initial resource allocation attempt is considered to have failed.

[0040] Specifically, this step attempts to perform rapid one-to-one resource matching without disrupting existing tasks. Based on the current resource distribution (i.e., a snapshot of the allocation relationships of all tasks at the previous moment), the system executes the following logic: Find a suitable one among the available GPU resources. A set of GPUs that meets the quantity requirements and hardware topology constraints (preferably the same switch, secondarily the same node).

[0041] If found, a new allocation relationship is determined for the set, the initial allocation is deemed successful, and the process ends.

[0042] If not found, the initial resource allocation attempt is considered a failure. This indicates a resource fragmentation problem: the total amount of idle resources may be sufficient, but their distribution cannot form contiguous blocks that satisfy the topological constraints of the new task.

[0043] As a lightweight pre-filter, it efficiently handles scenarios with sufficient and ideal resource distribution. Its failure is a necessary condition to trigger subsequent global reordering optimization (involving large-scale adjustments to allocation relationships).

[0044] S203. If the initial resource allocation attempt fails, construct a global scheduling task set that includes the target AI task and existing transferable tasks in the cluster.

[0045] When fast matching fails, the system switches to global optimization mode. This step defines the scope of tasks to be optimized: Determine a global scheduling task set T. This set includes not only the target AI task for the new request, but also all existing tasks in the cluster that are marked as portable. Small model tasks are typically considered portable candidates due to their flexible resource requirements and insensitivity to topology; while some large model tasks may also be included if their priority allows.

[0046] By fixing non-optimized objects, tasks that are explicitly not involved in reordering (such as high-priority or stateful services) will have their existing allocation state fixed, meaning it will remain unchanged throughout the entire reordering process. This reduces the scale of the optimization problem and ensures the stability of critical business operations.

[0047] By clearly defining the boundary (set T) and fixed part of the optimization problem, the scheduling problem is transformed from finding vacancy for new tasks to finding a better global layout for all tasks in set T, thus defining a clear optimization object for building a mathematical model.

[0048] S204. Using all tasks in the global scheduling task set as the objects to be scheduled, establish a resource rearrangement model with the optimization objective of minimizing the number of task migrations.

[0049] This step is the core of intelligent decision-making, transforming the scheduling problem into a multi-objective integer linear programming problem. The system establishes a mathematical model for optimization: To describe the final resource consumption of the task, a binary decision variable is defined. (Indicates whether task t is ultimately assigned to GPU g on node n) and auxiliary variables (Indicates whether task t ultimately occupies node n).

[0050] Next, define the core constraints to ensure the physical feasibility of the solution and the requirements of the task: Constraint 1: GPU exclusivity constraint, ensuring that any physical GPU can be occupied by at most one task at a time, preventing resource conflicts. Its expression is as follows:

[0051] in, This represents a specific AI task. This represents the global set of scheduled tasks, including the target AI task and all existing transferable tasks that are considered for reordering. This is a binary decision variable, taking the value 0 or 1. When the value is 1, it means that task t is assigned to GPU g on node n; when the value is 0, it means that it is not assigned. Represents a physical computing node. This represents the set of all physical computing nodes in the cluster. This refers to a specific GPU device. This represents the collection of all GPU devices within a compute node.

[0052] This constraint states that for any GPU g in any node n, the sum of all task allocation decisions for it must be less than or equal to 1. This ensures that any physical GPU can be allocated to at most one task at any given time, thus preventing conflicts where multiple tasks compete for the same physical resource. It is a fundamental device-level constraint that guarantees the feasibility of resource allocation.

[0053] Constraint 2: GPU quantity constraint for each task, ensuring that the final number of GPUs obtained for each task t is exactly equal to its requirement. Its expression is as follows:

[0054] in, This represents the number of GPUs requested by task t, which is a known input parameter.

[0055] This constraint states that for any task t, the sum of its allocation decisions across different nodes and different GPUs must be exactly equal to the number of GPUs requested by that task. This ensures that the resource requirements of each task are strictly met, which is the core functional constraint of the scheduling model.

[0056] Constraint 3: Cross-node restrictions, implemented through auxiliary variables. Ensure consistency in allocation relationships. Its expression is as follows:

[0057] in, This is an auxiliary binary variable, taking the value 0 or 1. A value of 1 indicates that task t occupies node n; a value of 0 indicates that it is not occupied.

[0058] This constraint is an auxiliary variable. Definitional constraints. It establishes the task-node occupancy variable. Assigning variables to task-node-GPU The logical relationship between them is as follows: Only when task t has at least one GPU allocated on node n (i.e., there exists some g such that...) When =1), The value can be 1. Conversely, if =0, then task t cannot be allocated any GPU on node n (all corresponding GPUs). (Must be 0). This constraint provides the basis for subsequent determinations of whether a task crosses nodes and for calculating cross-node migration penalties.

[0059] Constraint 4: Smaller models do not participate in rearrangement, and their expression is as follows:

[0060] in, This indicates whether task t was already assigned to GPU g on node n at the previous time before the reordering (i.e., the current resource distribution state). This is a known historical state parameter. This represents a subset of all small model tasks, which typically have low resource requirements and are insensitive to hardware topology.

[0061] This constraint means that for all sets of small model tasks... The task, its rearranged allocation decision It must be rearranged from its original state. Completely consistent. This essentially fixes these types of tasks in their original positions, preventing them from participating in this reordering optimization. This constraint simplifies the size of the optimization problem and ensures the stability of high-priority or state-sensitive small model services.

[0062] Topology constraints: For large model tasks that require the same switch, constraints are added to force all allocated GPUs to be on the same switch.

[0063] Then, a hierarchical optimization objective function is defined, balancing migration cost and topology optimization: Establish hierarchical optimization objectives, and ensure the optimization order by assigning different priority weights (W1>>W2>>W3): First priority: Minimize the number of cross-node migrations. Penalize tasks for moving between different nodes, as cross-node migrations are typically the most expensive.

[0064] Second priority: Minimize changes in GPU granularity allocation. After achieving the first objective, it is further desirable for tasks to use the same GPUs they originally occupied as much as possible, reducing the disturbance of resource adjustments within the same node.

[0065] Third priority: Optimize the layout within the switch. While meeting the first two objectives, encourage tasks (especially large model tasks) to concentrate resources within fewer switches to optimize physical topology and improve communication efficiency.

[0066] The objective function is defined as follows:

[0067] in, This represents the overall objective function value, with the goal of minimizing Z; The weight represents the penalty for cross-node changes, used to control the cost of migrating tasks between nodes; This represents the weight of the GPU change penalty, used to control the cost of adjusting GPU resources within the same node for a task; This indicates the weight of the Switch crossing penalty, which is used to control the penalty for tasks crossing different PCIe switch units (Switches); The binary decision variable (0 or 1) indicates whether task t crosses switch s, where s represents a specific PCIe switch and S represents the set of all PCIe switches in the cluster.

[0068] Complex, conflict-ridden scheduling decisions (meeting demand, reducing migration, optimizing topology) are formalized into a mathematical optimization problem with clear constraints and hierarchical objectives. This allows the system to compute the target resource distribution scheme (i.e., the optimal values ​​of all decision variables) by calling a standard optimization solver. This scheme mathematically guarantees the feasibility of resource allocation and systematically balances the need to deploy new tasks with the cost of migrating existing tasks, forming the core of achieving intelligent, automated, and global optimization.

[0069] S205. Solve the resource rearrangement model to obtain the target resource distribution scheme.

[0070] The integer linear programming model established in S204 is solved using an optimization solver (such as Gurobi, CPLEX, or the open-source tool OR-Tools). The solver takes the decision variables, constraints, and objective function in the model as input and outputs the optimal or feasible solution that satisfies all constraints, i.e., the target resource distribution scheme.

[0071] If the resource rearrangement model has no solution, then adjust the parallel execution configuration of the existing tasks based on the historical load data of existing tasks of the same type as the target AI task in the global scheduling task set; update the resource requirements of existing tasks in the global scheduling task set based on the adjusted parallel execution configuration; and rebuild the resource rearrangement model using the updated global scheduling task set.

[0072] By solving mathematical models, the system systematically finds the optimal or feasible resource allocation scheme under all constraints; it has the ability to adaptively adjust when there is no solution, thereby improving the scheduling success rate.

[0073] S206. Compare the target resource distribution plan with the current resource distribution status, generate a resource adjustment operation, and migrate the cluster resource status from the current distribution to the target distribution.

[0074] The system compares the target resource distribution plan with the current resource distribution status to obtain the first comparison result. Based on this result, the system can identify three types of operations that need to be performed: newly created task instances (corresponding to newly submitted tasks), old task instances that need to be deleted (corresponding to tasks terminated due to adjustments), and task instances that need to be migrated (corresponding to tasks whose running location needs to be changed). To ensure resource consistency and operational reliability during the migration process, the system will strictly follow the order of "first performing task instance deletion and creation, then performing task instance migration" to execute these resource adjustment operations sequentially. This order can avoid resource contention and state conflicts.

[0075] Specifically, when the computing devices allocated to the task instance to be migrated need to change their physical connection topology within the same computing node, the execution of task instance migration includes: deleting the currently running instance of the task instance to be migrated; determining the new physical connection order of the computing devices for the task instance to be migrated according to the target resource distribution scheme; and creating and running a new instance of the task instance to be migrated according to the new physical connection order of the computing devices.

[0076] The mathematical solutions are transformed into executable cluster operation instructions, enabling a safe and orderly transition from the current state to the target state. This ensures resource consistency and task availability during state transitions; avoids resource contention and deadlocks through operation sequencing; and supports fine-grained topology adjustments to adapt to the hardware topology-sensitive requirements of high-performance AI tasks. Based on the above, the technical solution proposed in this application constructs a complete closed loop from intelligent decision-making to precise execution. Its systematic nature, adaptability, and efficiency constitute the core features and significant advantages that distinguish it from existing static scheduling schemes. Specifically, at the technical level, this application transforms the complex resource orchestration problem into a mathematical problem that can be solved precisely by introducing an integer linear programming model with topological constraints and a hierarchical optimization objective. At the engineering level, a modular system architecture and an ordered sequence of operation execution ensure that the optimization results can be safely and reliably implemented. At the application level, adaptive retry and configuration adjustment mechanisms improve the success rate and resource utilization efficiency of deploying complex AI tasks in highly fragmented and dynamically changing cloud-native environments.

[0077] The above text combined Figures 1 to 3 The optimization method for dynamic migration and resource rearrangement of cloud-native AI tasks provided in the embodiments of this application has been described in detail. The apparatus and device provided in the embodiments of this application will be described below with reference to the accompanying drawings.

[0078] This application also provides an optimization device for dynamic migration and resource rearrangement of cloud-native AI tasks, such as... Figure 4As shown in the figure, this is a schematic diagram of an optimization device for dynamic migration and resource rearrangement of cloud-native AI tasks provided in an embodiment of this application. The device includes: The acquisition module 301 is used to acquire the resource requirements and hardware topology constraints of the target AI task in response to a resource scheduling request for the target AI task. The resource allocation module 302 is used to perform an initial resource allocation attempt for the target AI task based on the current resource distribution status of the cluster and the resource requirements and hardware topology constraints of the target AI task. The resource rearrangement module 303 is used to construct a global scheduling task set including the target AI task and existing migrateable tasks in the cluster if the initial resource allocation attempt fails; to establish a resource rearrangement model with minimizing the number of task migrations as the optimization objective, using all tasks in the global scheduling task set as the scheduling objects; to solve the resource rearrangement model to obtain the target resource distribution scheme; The resource adjustment module 304 is used to compare the target resource distribution scheme with the current resource distribution status, generate a resource adjustment operation, and migrate the cluster resource status from the current distribution to the target distribution.

[0079] In some possible implementations, the resource allocation module 302 is specifically used to search for a set of idle computing devices in the cluster that meet the conditions based on the resource requirements and the hardware topology constraints; if a set of idle computing devices that meet the conditions is found, the target AI task is allocated to the set of idle computing devices that meet the conditions; if no set of idle computing devices that meet the conditions is found, the initial resource allocation attempt is determined to have failed.

[0080] In some possible implementations, the hardware topology constraints include requiring computing devices to be located under the same physical switching unit; if it is not possible to satisfy the requirement that computing devices be located under the same physical switching unit, the constraint degenerates into requiring computing devices to be located within the same computing node.

[0081] In some possible implementations, the device further includes: The reconstruction module is used to, if the resource rearrangement model has no solution, adjust the parallel execution configuration of the existing tasks based on the historical load data of existing tasks of the same type as the target AI task in the global scheduling task set; update the resource requirements of the existing tasks in the global scheduling task set based on the adjusted parallel execution configuration; and rebuild the resource rearrangement model using the updated global scheduling task set.

[0082] In some possible implementations, the resource adjustment module 304 is specifically used to compare the target resource distribution scheme with the current resource distribution state to obtain a first comparison result; based on the first comparison result, determine the new task instances to be created, the old task instances to be deleted, and the task instances to be migrated; and execute the resource adjustment operation serially in the order of first deleting and creating task instances, and then migrating task instances.

[0083] In some possible implementations, when the computing devices allocated to the task instance to be migrated need to change their physical connection topology within the same computing node, the resource adjustment module 304 is specifically used to delete the current running instance of the task instance to be migrated; determine the new physical connection order of the computing devices for the task instance to be migrated according to the target resource distribution scheme; and create and run a new instance of the task instance to be migrated based on the new physical connection order of the computing devices.

[0084] In some possible implementations, the resource requirements and hardware topology constraints include the number of computing devices required for the target AI task and the required interconnect topology between the computing devices.

[0085] The optimization apparatus for dynamic migration and resource rearrangement of cloud-native AI tasks according to the embodiments of this application can correspond to the execution of the method described in the embodiments of this application, and the other operations and / or functions of each module / unit of the optimization apparatus for dynamic migration and resource rearrangement of cloud-native AI tasks are respectively for implementing Figure 3 For the sake of brevity, the corresponding processes of each method in the illustrated embodiments will not be described in detail here.

[0086] This application also provides a computing device. For example... Figure 5 As shown in the figure, this is a schematic diagram of a computing device provided in an embodiment of this application. The computing device 400 includes a bus 401, a processor 402, a communication interface 403, and a memory 404. The processor 402, the memory 404, and the communication interface 403 communicate with each other via the bus 401.

[0087] Bus 401 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 5 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0088] Processor 402 can be any one or more of the following processors: central processing unit (CPU), graphics processing unit (GPU), microprocessor (MP), or digital signal processor (DSP).

[0089] The communication interface 403 is used for communication with external devices. For example, if the computing device is a first switch, the communication interface 403 can be used for communication between the first switch and a first user terminal, or for communication between the first switch and a second switch.

[0090] Memory 404 may include volatile memory, such as random access memory (RAM). Memory 404 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0091] The memory 404 stores executable code, and the processor 402 executes the executable code to perform the aforementioned optimization method for dynamic migration and resource rearrangement of cloud-native AI tasks.

[0092] Specifically, in achieving Figure 4 In the case of the illustrated embodiment, and Figure 4 When the modules or units of the optimization device for dynamic migration and resource rearrangement of cloud-native AI tasks described in the embodiments are implemented in software, the execution... Figure 4 The software or program code required for the functions of each module / unit can be partially or entirely stored in memory 404. Processor 402 executes the program code corresponding to each unit stored in memory 404, and executes the aforementioned optimization method for dynamic migration and resource rearrangement of cloud-native AI tasks.

[0093] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute the aforementioned optimization method for dynamic migration and resource rearrangement of cloud-native AI tasks.

[0094] This application also provides a computer program product comprising one or more computer instructions. When the computer instructions are loaded and executed on a computing device, all or part of the processes or functions described in this application are generated.

[0095] The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, or data center to another website, computer, or data center via wired (e.g., coaxial cable, fiber optic) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0096] When the computer program product is executed by a computer, the computer performs any of the aforementioned optimization methods for dynamic migration and resource rearrangement of cloud-native AI tasks. The computer program product can be a software installation package; when any of the aforementioned optimization methods for dynamic migration and resource rearrangement of cloud-native AI tasks needs to be used, the computer program product can be downloaded and executed on the computer.

[0097] The descriptions of the processes or structures corresponding to the above figures each have their own emphasis. For parts of a process or structure that are not described in detail, please refer to the relevant descriptions of other processes or structures.

[0098] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be covered within the scope of protection of this application.

Claims

1. An optimization method for dynamic migration and resource rearrangement of cloud-native AI tasks, characterized in that, The method includes: In response to a resource scheduling request for a target AI task, the resource requirements and hardware topology constraints of the target AI task are obtained. Based on the current resource distribution status of the cluster and the resource requirements and hardware topology constraints of the target AI task, an initial resource allocation attempt is performed for the target AI task. If the initial resource allocation attempt fails, a global scheduling task set is constructed, which includes the target AI task and existing transferable tasks in the cluster. Using all tasks in the global scheduling task set as the scheduling objects, a resource rearrangement model is established with the optimization objective of minimizing the number of task migrations. The resource rearrangement model includes node-level constraints and device-level constraints based on the cluster hardware topology. Solve the resource rearrangement model to obtain the target resource distribution scheme; The target resource distribution scheme is compared with the current resource distribution status, and a resource adjustment operation is generated to migrate the cluster resource status from the current distribution to the target distribution.

2. The method according to claim 1, characterized in that, The initial resource allocation attempt for the target AI task includes: Based on the resource requirements and hardware topology constraints, find a set of idle computing devices in the cluster that meet the conditions; If a set of available computing devices that meet the criteria is found, the target AI task will be assigned to that set of available computing devices. If no set of available computing devices that meet the criteria is found, the initial resource allocation attempt is deemed to have failed.

3. The method according to claim 1, characterized in that, The hardware topology constraints include the requirement that computing devices be located under the same physical switching unit; if it is not possible to satisfy the requirement that computing devices be located under the same physical switching unit, the constraint degenerates into the requirement that computing devices be located within the same computing node.

4. The method according to claim 1, characterized in that, The method further includes: If the resource rearrangement model has no solution, then the parallel execution configuration of the existing tasks is adjusted based on the historical load data of existing tasks of the same type as the target AI task in the global scheduling task set. Based on the adjusted parallel execution configuration, update the resource requirements of the existing tasks in the global scheduling task set; The resource rearrangement model is rebuilt using the updated global scheduling task set.

5. The method according to claim 1, characterized in that, The step of comparing the target resource distribution scheme with the current resource distribution state and generating a resource adjustment operation includes: The target resource distribution scheme is compared with the current resource distribution status to obtain a first comparison result; Based on the first comparison result, determine the new task instances that need to be created, the old task instances that need to be deleted, and the task instances that need to be migrated; The resource adjustment operations are performed sequentially, first deleting and creating task instances, and then migrating task instances.

6. The method according to claim 5, characterized in that, When the computing device assigned to the task instance to be migrated needs to change its physical connection topology within the same computing node, the execution of task instance migration includes: Delete the currently running instance of the task instance that needs to be migrated; Based on the target resource distribution scheme, determine the new physical connection order of computing devices for the task instances that need to be migrated; Based on the new physical connection order of the computing devices, create and run a new instance of the task instance that needs to be migrated.

7. The method according to claim 1, characterized in that, The resource requirements and hardware topology constraints include the number of computing devices required for the target AI task and the required interconnection topology between the computing devices.

8. An optimization device for dynamic migration and resource rearrangement of cloud-native AI tasks, characterized in that, The device includes: The acquisition module is used to acquire the resource requirements and hardware topology constraints of the target AI task in response to a resource scheduling request for the target AI task. The resource allocation module is used to perform an initial resource allocation attempt for the target AI task based on the current resource distribution status of the cluster and the resource requirements and hardware topology constraints of the target AI task. The resource rearrangement module is used to construct a global scheduling task set, including the target AI task and existing migrateable tasks in the cluster, if the initial resource allocation attempt fails; using all tasks in the global scheduling task set as the scheduling objects, a resource rearrangement model is established with the optimization objective of minimizing the number of task migrations. The resource rearrangement model includes node-level constraints and device-level constraints based on the cluster hardware topology; the resource rearrangement model is solved to obtain the target resource distribution scheme. The resource adjustment module is used to compare the target resource distribution scheme with the current resource distribution status, generate a resource adjustment operation, and migrate the cluster resource status from the current distribution to the target distribution.

9. A computing device, characterized in that, Including memory and processor; The memory stores one or more computer programs, the one or more computer programs including instructions; when the instructions are executed by the processor, the computing device performs the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program for performing the method as described in any one of claims 1 to 7.

Images