GPU resource pool dynamic management system and method based on container platform

Abstract

The invention relates to the technical field of GPU resource management, and discloses a GPU resource pool dynamic management system and method based on a container platform, a scheduling demand is obtained by analyzing a user resource request, and resource matching and scheduling decision making are performed based on the demand and a real-time cluster resource state. And issuing and executing a resource pool management instruction according to the decision, and finally aggregating operation results to synchronously update the cluster state. The method aims at solving the problems that in the prior art, a GPU resource pool management mode is rigid, a scheduling strategy is solidified, and heterogeneous resources are difficult to manage in a unified mode. The GPU resource pool can be dynamically created, updated or released as required by constructing a closed-loop management process which is driven by a strategy and from request analysis to state synchronization, so that unified abstraction and scheduling of heterogeneous GPU resources are realized, and the flexibility of resource management, the system response efficiency and the adaptability to diversified business scenes are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of GPU resource management, and more specifically, to a GPU resource pool dynamic management system and method based on a container platform. BACKGROUND

[0002] With the rapid development of technologies such as artificial intelligence, machine learning, and high-performance computing, graphics processing units (GPUs) have become an indispensable core acceleration hardware in modern computing architectures due to their powerful parallel computing capabilities. In the context of the widespread application of cloud-native and containerization technologies, how to efficiently and flexibly manage and schedule GPU resources has become a key challenge in improving the overall performance and resource utilization of systems.

[0003] Currently, mainstream container platforms provide basic scheduling capabilities for GPU resources, but generally adopt a single static resource pool management mode. This mode pre-includes all available GPU resources in the cluster into a centralized static pool for unified management. Although it is relatively simple to implement and maintain, its resource allocation method is relatively rigid, lacking the fine-grained control capability of dynamically creating, updating, or releasing resources on demand. When faced with diverse and dynamically changing business loads, this mode is difficult to quickly respond and create or adjust matching GPU resource pools according to the specific needs of different workloads, resulting in insufficient resource adaptability and response efficiency.

[0004] Further, existing GPU scheduling mechanisms mostly hard-code scheduling strategies within the scheduler, and users cannot flexibly define or adjust scheduling logic according to actual business needs. For example, systems generally lack differentiated support for different task types (such as high memory demand tasks, low latency inference tasks, or communication-intensive training tasks), and users also have difficulty specifying advanced scheduling strategies such as node affinity, specific GPU models, or topology structures through configuration. This "one-size-fits-all" rigid scheduling approach significantly limits the intelligence level of resource scheduling and the overall adaptability of the system when faced with complex scenarios such as multi-tenancy or heterogeneous GPU clusters.

[0005] In addition, in actual deployment, enterprises often mix and deploy heterogeneous GPU devices from different manufacturers or different models from the same manufacturer in order to balance cost and performance. However, due to significant differences in hardware specifications, driver interfaces, and scheduling mechanisms among different types of GPUs, there is currently a lack of a unified GPU resource pool management mechanism. This results in the need for upper-layer applications to explicitly perceive and adapt to specific GPU types when deploying, which not only increases the complexity of deployment and operation and maintenance, but also limits the flexibility of application migration between different hardware environments, reducing the universality and scalability of the system SUMMARY

[0006] To solve the above technical problems, the present application is proposed. Embodiments of the present application propose a GPU resource pool dynamic management system and method based on a container platform, aiming to improve the flexibility and system response efficiency of GPU resource management, support dynamically configurable scheduling strategies to meet diversified business needs, and realize unified abstraction and centralized management of heterogeneous GPU resources, thereby enhancing the adaptability, compatibility and universality of the system.

[0007] According to an aspect of the present application, a GPU resource pool dynamic management method based on a container platform is provided, comprising the following steps: performing request analysis on a user resource request to obtain an analyzed scheduling requirement; performing resource matching and scheduling decision based on the analyzed scheduling requirement and the cluster resource state to obtain a scheduling decision; performing resource pool management instruction issuing and execution based on the scheduling decision to obtain an operation result; and performing cluster state aggregation and metadata synchronization based on the operation result to obtain an updated cluster resource state.

[0008] In a possible implementation, performing request analysis on a user resource request to obtain an analyzed scheduling requirement comprises: performing legality verification on the request analysis of the user resource request to obtain a verified user resource request; extracting core demand parameters from the verified user resource request to obtain the number of GPUs extracted; extracting a GPU scheduling strategy from the verified user resource request, and performing scheduling strategy-filter mapping and generation on the GPU scheduling strategy to obtain a filter condition list; and performing object aggregation on the extracted number of GPUs and the filter condition list to obtain the analyzed scheduling requirement.

[0009] In a possible implementation, performing resource matching and scheduling decision based on the analyzed scheduling requirement and the cluster resource state to obtain a scheduling decision comprises: performing candidate GPU resource identification on the cluster resource state based on the filter condition list in the analyzed scheduling requirement to obtain a candidate GPU list; performing resource adequacy determination on the candidate GPU list based on the extracted number of GPUs in the analyzed scheduling requirement to obtain an adequacy flag; in response to the adequacy flag being true, performing scheduling scheme scoring and sorting on the candidate GPU list based on the allocation strategy and the extracted number of GPUs to obtain a sorted scheme list; and performing optimal GPU set selection and resource pool naming on the sorted scheme list based on the request context to obtain the scheduling decision.

[0010] In a possible implementation, the optimal GPU set selection and resource pool naming are performed on the ranked scheme list based on the request context to obtain the scheduling decision, including: performing the optimal GPU set selection and resource pool naming on the ranked scheme list based on the request context to obtain a selected GPU list and a generated resource pool name; and performing scheduling decision aggregation on the selected GPU list and the generated resource pool name to obtain the scheduling decision.

[0011] In a possible implementation, the scheduling scheme scoring and ranking are performed on the candidate GPU list based on the allocation strategy and the extracted GPU quantity in response to the sufficiency flag being true to obtain the ranked scheme list, including: inputting the extracted GPU quantity and the candidate GPU list into a combination generator; applying a combination mathematical algorithm by the combination generator to process the extracted GPU quantity and the candidate GPU list to obtain a possible scheme list; performing scheme quantity scoring and structured packaging on each possible scheme in the possible scheme list based on the allocation strategy to obtain an unranked scored scheme list; and performing scheme priority ranking on the unranked scored scheme list to obtain the ranked scheme list.

[0012] In a possible implementation, the allocation strategy includes a packing strategy and a scattering strategy.

[0013] In a possible implementation, the scheme quantity scoring and structured packaging are performed on each possible scheme in the possible scheme list based on the allocation strategy to obtain the unranked scored scheme list, including: extracting a first possible scheme from the possible scheme list; extracting a unique node ID number from the first possible scheme; if the allocation strategy is the packing strategy, the score of the first possible scheme is 1 divided by the unique node ID number; and if the allocation strategy is the scattering strategy, the score of the first possible scheme is the unique node ID number.

[0014] According to another aspect of the present application, a GPU resource pool dynamic management system based on a container platform is provided, including: a resource pool scheduling strategy configuration module configured to receive a resource request defined by a user and containing a GPU resource requirement and a scheduling strategy; a resource pool scheduling coordination control module connected to the resource pool scheduling strategy configuration module and configured to perform resource matching and scheduling decision making according to the resource request and a preset cluster resource state, and generate an instruction for dynamically managing a GPU resource pool; and a resource pool dynamic management module connected to the resource pool scheduling coordination control module and configured to execute the dynamic management instruction to perform pool management on one or more GPU resources, thereby dynamically creating, updating, or releasing the GPU resource pool, and report a GPU resource state to update the cluster resource state.

[0015] Compared with the prior art, the GPU resource pool dynamic management system and method based on a container platform provided by the application firstly breaks the rigid mode of the traditional static resource pool by dynamically creating, updating and releasing the GPU resource pool on demand, greatly improves the flexibility of GPU resource management and the response efficiency of the system to dynamic business load, and thus optimizes the resource utilization rate. Secondly, the dynamically configurable resource scheduling strategy mechanism is introduced, so that the system can accurately adapt to various differentiated business scenarios such as high memory tasks, low delay inference or communication intensive training, significantly enhancing the intelligent level of task scheduling and the business adaptation ability of the system. Finally, through the unified abstraction and centralized management of heterogeneous GPU resources, the differences of the underlying hardware are effectively shielded, not only reducing the deployment and operation and maintenance complexity of the application, but also greatly improving the compatibility, universality and migration ability of the application between different hardware platforms. BRIEF DESCRIPTION OF DRAWINGS

[0016] The above and other objects, features and advantages of the present application will become more apparent from the following detailed description of embodiments of the present application, when taken in conjunction with the accompanying drawings. The drawings provided in the specification and the embodiments of the present application together serve to explain constituents of the present application and to provide further understanding of the present application, and do not limit the present application. In the drawings, like reference numerals refer to like parts or steps throughout the several views.

[0017] Figure 1 FIG. 1 illustrates a schematic flowchart of a GPU resource pool dynamic management method based on a container platform according to an embodiment of the present application.

[0018] Figure 2 FIG. 1 illustrates a schematic flowchart of a GPU resource pool dynamic management method based on a container platform according to an embodiment of the present application.

[0019] Figure 3 FIG. 1 illustrates a schematic flowchart of a GPU resource pool dynamic management method based on a container platform according to an embodiment of the present application.

[0020] Figure 4 FIG. 1 illustrates a schematic architecture diagram of a GPU resource pool dynamic management system based on a container platform according to an embodiment of the present application.

[0021] Figure 5 FIG. 1 illustrates a schematic flowchart of a GPU resource pool dynamic management method based on a container platform according to an embodiment of the present application.

[0022] Figure 6 FIG. 1 illustrates a schematic flowchart of a GPU resource pool dynamic management method based on a container platform according to an embodiment of the present application.

[0023] Figure 7Fig. 1 shows a schematic flow chart of a GPU resource pool dynamic expansion process according to an embodiment of the present application.

[0024] Figure 8 Fig. 2 shows a schematic flow chart of a GPU resource pool dynamic scaling process according to an embodiment of the present application.

[0025] Figure 9 Fig. 3 shows a schematic flow chart of a GPU resource pool dynamic release process according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] Hereinafter, example embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part but not all of the embodiments of the present application, and the present application can be implemented in many different forms. Therefore, the attached drawings should not be used to limit and define the present application, and the present application should cover all changes falling within the scope of the appended claims and their equivalents.

[0027] Figure 1 Fig. 4 shows a schematic flow chart of a GPU resource pool dynamic management system and method based on a container platform according to an embodiment of the present application. As shown in Figure 1 the present application provides a GPU resource pool dynamic management method based on a container platform, comprising the following steps: S1, performing request analysis on a user resource request to obtain an analyzed scheduling requirement; S2, performing resource matching and scheduling decision based on the analyzed scheduling requirement and a cluster resource state to obtain a scheduling decision; S3, performing resource pool management instruction issuing and execution based on the scheduling decision to obtain an operation result; and S4, performing cluster state aggregation and metadata synchronization based on the operation result to obtain an updated cluster resource state.

[0028] For example, in step S1, the user resource request is analyzed to obtain an analyzed scheduling requirement. It should be understood that, considering that the prior art generally lacks flexible request expression and analysis mechanism, resulting in fixed scheduling strategy, it is difficult to meet the differentiated needs of different business scenarios. The present application successfully builds a bridge between the user's high-level intention and the system's underlying execution logic by introducing a configurable request interface and a refined analysis process, which is the premise and foundation for realizing strategy-driven intelligent and dynamic resource management.

[0029] In one embodiment, as Figure 2As shown, the user resource request is parsed to obtain a parsed scheduling requirement, including: S11, the user resource request is parsed to obtain a verified user resource request; S12, the core demand parameter of the verified user resource request is extracted to obtain an extracted GPU quantity; S13, the GPU scheduling strategy is extracted from the verified user resource request, and the scheduling strategy-filter mapping and generation are performed on the GPU scheduling strategy to obtain a filter condition list; S14, the extracted GPU quantity and the filter condition list are aggregated to obtain the parsed scheduling requirement.

[0030] Specifically, first, request receiving and legality verification are performed to ensure that the request entering the system is in correct format and meets the basic business rules. After receiving the original user resource request data, syntax and semantic verification is performed, for example, checking whether it is a legal JSON format, verifying whether the "GPU_Count" field exists and its value is a valid positive integer, and checking whether all strategy fields in the "Scheduling_Strategy" object are legal strategies supported by the system.

[0031] Next, the core demand parameter of the verified user resource request is extracted to obtain an extracted GPU quantity. This process is specifically responsible for extracting the explicit, non-strategic core resource quantity in the request. The parameter extractor will locate the "GPU_Count" field in the verified user resource request data, read its value and convert it into an independent integer variable, i.e. the extracted GPU quantity (Extracted_GPU_Count).

[0032] Then, the GPU scheduling strategy is extracted from the verified user resource request, and the scheduling strategy-filter mapping and generation are performed on the GPU scheduling strategy to obtain a filter condition list. Specifically, the strategy iterator traverses each key-value pair in the "Scheduling_Strategy" object, and for each key-value pair, the filter generator uses the strategy key (such as "GPU_Type") to look up the corresponding filter function name (such as "filter_by_gpu_type") in a predefined "Strategy_To_Filter_Mapper" table, and then combines the strategy value (such as "NVIDIA-A100") to dynamically construct a complete filter condition string that can be used for subsequent resource screening, for example "filter_by_gpu_type('NVIDIA-A100')". All generated filter condition strings are collected to form a "Generated_Filter_Conditions" list.

[0033] Finally, the extracted GPU count and filter condition list are object-aggregated to obtain the resolved scheduling requirement. The object builder creates a new, empty resolved scheduling requirement object, assigns the extracted GPU count obtained in the second step to the "Required_GPU_Count" field of the object, and assigns the filter condition list obtained in the third step to the "Filter_Conditions" field of the object, and finally outputs a complete, structured resolved scheduling requirement data object.

[0034] To illustrate this process more specifically, a specific embodiment is now provided, in which a user resource request is first received, with the content being: {"GPU_Count":"2", "Scheduling_Strategy":{"GPU_Type":"NVIDIA-RTX-4090", "Node_Label":"high-bandwidth-node"}}. In the request resolution step, first, the request passes the legality check. Then, the core requirement parameter extraction sub-step extracts the extracted GPU count as an integer 2. Subsequently, in the scheduling strategy-filter mapping and generation sub-step, the "Scheduling_Strategy" object is traversed: for the key-value pair "GPU_Type":"NVIDIA-RTX-4090", the filter condition string "filter_by_gpu_type('NVIDIA-RTX-4090')" is generated; for the key-value pair "Node_Label":"high-bandwidth-node", the filter condition string "filter_by_node_label('high-bandwidth-node')" is generated. Finally, in the object aggregation and construction sub-step, the extracted count 2 and the two generated filter condition strings are aggregated, and the final output "resolved scheduling requirement" is: {"Required_GPU_Count":2, "Filter_Conditions":["filter_by_gpu_type('NVIDIA-RTX-4090'), "filter_by_node_label('high-bandwidth-node')"]}.

[0035] Exemplarily, in step S2, resource matching and scheduling decision are made based on the parsed scheduling requirements and the cluster resource state to obtain a scheduling decision. It should be understood that, considering the fundamental problems of traditional static resource pool structure and fixed scheduling logic, the application innovatively introduces a resource pool dynamic creation mechanism driven by scheduling strategy, which can intelligently match and decide in the global resource view in real time according to the resource request of the user and the configured scheduling strategy, thereby dynamically triggering the subsequent GPU resource pool creation, update or release operation. It deeply integrates the global resource state awareness capability and the scheduling execution mechanism, breaks the separation between resource scheduling and resource management in traditional systems, and builds a highly linked scheduling closed loop, which is the fundamental guarantee for realizing dynamic matching of resource supply and task demand.

[0036] In one embodiment, as shown in Figure 3 Scheduling decision is made based on the parsed scheduling requirements and the cluster resource state, including: S21, based on the filter condition list in the parsed scheduling requirements, candidate GPU resources are identified from the cluster resource state to obtain a candidate GPU list; S22, based on the extracted GPU quantity in the parsed scheduling requirements, resource sufficiency of the candidate GPU list is determined to obtain a sufficiency flag; S23, in response to the sufficiency flag being true, scheduling scheme scoring and sorting of the candidate GPU list are performed based on the allocation strategy and the extracted GPU quantity to obtain a sorted scheme list; S24, based on the request context, the optimal GPU set is selected from the sorted scheme list and the resource pool is named to obtain the scheduling decision.

[0037] Specifically, first, based on the filter condition list in the parsed scheduling requirements, candidate GPU resources are identified from the cluster resource state to obtain a candidate GPU list. In this process, the scheduler will traverse the global cluster resource state data, and for each GPU whose state is Free, the filter execution engine will apply all filter conditions in the Filter_Conditions list in the parsed scheduling requirements one by one for matching. If a GPU meets all the filter conditions, an object containing its complete context information (such as the node ID where it is located and its own GPU ID) will be added to a temporary candidate GPU list (Candidate_GPU_List).

[0038] Then, based on the extracted GPU count in the resolved scheduling requirement, a resource adequacy determination is performed on the candidate GPU list to obtain an adequacy flag. This is a decision gateway, the system will calculate the length of the candidate GPU list generated in the last step, and compare it with the "Required_GPU_Count" recorded in the resolved scheduling requirement. If the number of candidate GPUs is greater than or equal to the required number, it is determined that the resources are adequate, and a true adequacy flag is generated, and the process continues; otherwise, the scheduling fails, and the request may enter the waiting queue or be rejected.

[0039] Then, in response to the adequacy flag being true, the candidate GPU list is scored and ranked based on the allocation strategy and the extracted GPU count to obtain a ranked solution list. When the resources are adequate, the optimal solution needs to be selected from multiple possible GPU combinations. This process first inputs the extracted GPU count and the candidate GPU list into a combination generator, and then the combination generator applies a combination mathematical algorithm to process the extracted GPU count and the candidate GPU list to obtain a possible solution list. Subsequently, each possible solution in the possible solution list is scored and structured based on the allocation strategy to obtain an unsorted scored solution list. In one embodiment, the allocation strategy includes a packing strategy and a scattering strategy. Specifically, a first possible solution is extracted from the possible solution list, and a unique node ID number is extracted from the first possible solution. If the allocation strategy is the packing strategy, the score of the first possible solution is 1 divided by the unique node ID number, and this strategy tends to concentrate GPUs on a small number of nodes to achieve high performance; if the allocation strategy is the scattering strategy, the score of the first possible solution is the unique node ID number, and this strategy tends to scatter the load to more nodes to improve availability. After the scores of all possible solutions are calculated, the unsorted scored solution list is ranked in terms of solution priority, i.e. arranged in descending order of scores, to obtain a ranked solution list.

[0040] Finally, based on the request context, the ranked solution list is selected for the optimal GPU set and the resource pool naming to obtain the scheduling decision. Specifically, the highest scoring solution in the ranked solution list is directly selected, and the GPU combination in the solution is taken as the selected GPU list. At the same time, a globally unique generated resource pool name is generated by a name generator according to the user identity and other context information of the initiating request. Finally, by aggregating the selected GPU list and the generated resource pool name, a complete scheduling decision object containing the operation type, the resource pool name and the target GPU list is assembled and output.

[0041] To illustrate this series of complex matching and decision-making processes more specifically, a specific embodiment is now provided, in which the first received parsed scheduling request is for 2 GPUs of type “NVIDIA-A100” and located on nodes tagged as “high-perf”, while the system’s configured allocation policy is “PACK”. The current cluster resource state shows that there are 2 free A100 GPUs on node-01 (tagged with high-perf) and 1 free A100 GPU on node-02 (tagged with high-perf). First, after candidate GPU resources are identified, a “candidate GPU list” containing these 3 GPUs is obtained. Then, the resource sufficiency determination passes because 3 is greater than 2. Then, in the scheduling scheme scoring and sorting phase, three possible schemes are generated: scheme 1 is two GPUs on node-01, involving 1 node, with a score of 1.0 / 1 = 1.0; schemes 2 and 3 are each one GPU on node-01 and node-02, involving 2 nodes, with scores of 1.0 / 2 = 0.5 each. Since the allocation policy is a packing policy, the highest scoring scheme 1 is ranked first. Finally, in the optimal GPU set selection and resource pool naming phase, the two GPUs in scheme 1 are selected as the selected GPU list, and a unique resource pool name is generated according to the request context. The final output scheduling decision clearly indicates that the system creates a new resource pool on node-01 using the two selected GPUs.

[0042] Exemplarily, in step S3, resource pool management instruction issuing and execution are performed based on the scheduling decision to obtain an operation result. Specifically, by converting the scheduling decision into specific, executable node-level operation instructions, and through a distributed execution and result feedback mechanism, it is ensured that the decision is accurately implemented on the physical resources of the cluster. It works through the coordinated work of a centralized coordination control module and dynamic management modules distributed on each node to land the decision into real resource state changes, thus making it possible to dynamically create, update or release GPU resource pools.

[0043] Specifically, in the process of step S3, first, the scheduling decision data is received and the “Target_GPUs” in the decision are grouped by “Node_ID”. Then, it generates a specific node operation instruction (NodeOperationCommand) for each group, i.e. for each target node. The instruction contains a specific operation action (such as “ADD_TO_POOL”), a resource pool name obtained from the decision, and a list of GPU IDs on the node that need to be operated.

[0044] Subsequently, after receiving the node operation instruction, it first locally verifies the GPU ID list in the instruction to ensure that the GPUs exist and are in the correct state. After verification, it modifies the metadata of these GPU resources through the API of the container platform (such as the Kubernetes API Server) by adding a label or annotation. This step is the key operation to achieve resource pooling, which innovatively uses a hierarchical naming mechanism of "<uniform resource pool domain> / <user-defined namespace>" to take the resource pool name obtained from the decision as the key of the label and set a specific value (such as "true"), thereby logically classifying the GPU into the newly created resource pool. This mechanism not only masks the differences between underlying heterogeneous GPU resources, but also achieves naming isolation in a multi-user environment. After completing the operation on all specified GPUs on the node, an operation result data object is constructed, filling in the node's ID, the list of all successfully operated GPU IDs, and setting the status to "SUCCESS". If any step fails, the status is set to "FAILURE" with error information. Finally, the module sends the constructed "operation result" back to the resource pool scheduling coordination control module.

[0045] To illustrate this process more specifically, a specific embodiment is now provided, in which it is assumed that the scheduling decision generated in the previous step is {"Action":"CREATE_POOL","Pool_Name":"gpu-pool-domain / user-xyz-custom-pool","Target_GPUs":[{"Node_ID":"node-01","GPU_ID":"uuid-001"},{"Node_ID":"node-01","GPU_ID":"uuid-002"}]}. In the instruction issuing phase, the scheduling decision generated in the previous step is parsed, and it is found that both target GPUs are located in node-01. Therefore, it creates a node operation instruction for "node-01", which includes the operation "ADD_TO_POOL", the resource pool name "gpu-pool-domain / user-xyz-custom-pool", and the GPU list ["uuid-001", "uuid-002"]. In the instruction execution phase, after receiving the instruction, it adds a label "gpu-pool-domain / user-xyz-custom-pool:"true" to the two GPU resource objects with IDs "uuid-001" and "uuid-002" through the Kubernetes API. After the operation is successful, it constructs an "operation result": {"Node_ID":"node-01","Affected_GPU_IDs":["uuid-001","uuid-002"],"Status":"SUCCESS","ErrorMessage":null}, and reports it, thus completing the complete process from decision to execution.

[0046] Exemplarily, in step S4, the cluster state aggregation and metadata synchronization are performed based on the operation result to obtain an updated cluster resource state. It should be understood that, considering that the instruction execution is independently completed on the distributed nodes, the central scheduler must have a reliable mechanism to collect these scattered operation results and use them to update the global resource view it maintains. The purpose of this step is to feed back and solidify the actual changes of the physical resources to the central state library of the system by synchronizing the key metadata information in the cluster, so that the newly created or changed resource pool is visible and available to the entire system, and provides the latest and most accurate data basis for all subsequent scheduling decisions, and finally realizes a complete and self-consistent management closed loop from decision to execution to state feedback.

[0047] To safely and reliably synchronize the atomic operation results from each node into the global cluster state, the execution of step S4 includes the following process: First, it asynchronously listens to and receives operation result reports from all target nodes. After receiving all reports related to a certain scheduling decision, it aggregates and analyzes these reports. If the "Status" field of all reports is "SUCCESS", the system confirms that the entire resource pool management operation (such as creation) is successful, and enters the success path.

[0048] In the success path, each successful operation result report is traversed. According to the "Node_ID" and "Affected_GPU_IDs" (list of affected GPU IDs) provided in the report, the corresponding GPU object is accurately found and located in the internally maintained cluster resource state data structure. For each matched GPU object, its metadata is updated: the value of its "Pool_Membership" field is updated to the target resource pool name of this operation (the name comes from the initial scheduling decision), and the value of its "Allocated_Status" field is updated from "Free" to "IN_POOL" or "ALLOCATED", indicating that the GPU has been pooled and is in a non-idle state.

[0049] Conversely, if any "operation result" report with a status of "FAILURE" is received in the aggregation and analysis stage, it will be determined that the entire operation has failed, and the failure path, i.e. the rollback mechanism, will be triggered. In order to ensure the eventual consistency of the resource state, node operation instructions in the opposite direction (for example, "REMOVE_FROM_POOL" instructions corresponding to "ADD_TO_POOL" instructions) will be sent to all nodes that have successfully executed instructions in this operation, in order to undo all changes that have been made. This mechanism ensures that there is no intermediate state of partial pool creation or inconsistent state. Whether it is a successful submission or a successful rollback, when all operations are completed, the updated cluster resource state will be solidified. At this point, the new GPU resource pool has either been successfully created and is being uniformly managed by the system, ready for subsequent task scheduling, or all changes have been safely undone and the cluster state has returned to the state before the operation.

[0050] To illustrate this process more specifically, a specific embodiment is now provided, in which a operation result report {"Node_ID":"node-01","Affected_GPU_IDs":["uuid-001","uuid-002"],"Status":"SUCCESS","ErrorMessage":null} is first received. Since this is the only operation result involved in this decision, and its status is "SUCCESS", the success path is entered. In the internal cluster resource state data, the two GPU objects with IDs "uuid-001" and "uuid-002" under the "node-01" node are located. Then, it updates the "Pool_Membership" field of both GPU objects to "gpu-pool-domain / user-xyz-custom-pool", and updates their "Allocated_Status" fields from "Free" to "IN_POOL". After completing these in-memory data modifications, the state is persisted. At this point, the global resource view accurately reflects that the two GPUs now belong to a newly created resource pool named "gpu-pool-domain / user-xyz-custom-pool", and the entire dynamic creation process is thus closed loop.

[0051] In summary, the GPU resource pool dynamic management method based on the container platform provided by the present application has been illustrated. First, by supporting the dynamic creation, update and release of GPU resource pools on demand, the problem of rigid resource allocation caused by the static management mode in the prior art is effectively overcome, and the flexibility of resource scheduling and the response efficiency of the system are significantly improved. On this basis, the limitation of hard-coded scheduling logic in the current GPU scheduling mechanism is further broken, and by supporting dynamically configurable GPU resource scheduling strategies, users can flexibly define scheduling rules according to different business scenarios such as high memory, low latency inference or communication-intensive training, which not only improves the accuracy of task scheduling, but also greatly enhances the intelligent adaptability to diversified business types. This powerful strategy driving capability is based on the unified management and scheduling of heterogeneous GPU resources. Through unified abstraction and centralized management mechanism, the differences in hardware specifications and driver interfaces of GPUs from different manufacturers (such as NVIDIA, AMD, and Cambrian) are effectively shielded, so that the upper layer application does not need to perceive the underlying device, thereby simplifying the deployment and operation complexity, and significantly improving the compatibility, universality and maintainability of the system.

[0052] The application also provides a GPU resource pool dynamic management system based on a container platform, which is used to execute the GPU resource pool dynamic management method based on the container platform provided by the above-mentioned embodiments. Figure 4 As shown in the figure, the GPU resource pool dynamic management system based on the container platform comprises: a resource pool scheduling strategy configuration module, which is used to receive a resource request defined by a user and containing GPU resource requirements and scheduling strategies; a resource pool scheduling coordination control module, which is connected with the resource pool scheduling strategy configuration module and is used to make resource matching and scheduling decisions according to the resource request and a preset cluster resource state, and generate instructions for dynamically managing a GPU resource pool; and a resource pool dynamic management module, which is connected with the resource pool scheduling coordination control module and is used to execute the dynamic management instructions to perform pool management on one or more GPU resources, so as to dynamically create, update or release the GPU resource pool, and report a GPU resource state to update the cluster resource state.

[0053] Specifically, the resource pool scheduling strategy configuration module is a user interaction entrance and a policy definition front end of the GPU resource pool dynamic management system based on the container platform, and its core function is to receive a resource request defined by a user and containing GPU resource requirements and specific scheduling strategies. As one of the key innovative points of the GPU resource pool dynamic management system based on the container platform, the module is established to solve the fundamental problem that the existing GPU scheduling systems generally hard-code scheduling logic in the internal scheduler, thereby leading to rigid scheduling and lack of flexibility. In sharp contrast to the traditional technology, the module innovatively introduces a configurable GPU resource request and scheduling strategy interface, which supports the user to define resource requests and scheduling rules through flexible and diverse parameter configurations according to actual business requirements. The goal of this design is to achieve accurate matching between task requirements and resource allocation, thereby significantly improving the adaptability and intelligent level of scheduling of the system.

[0054] To achieve such a high degree of customization capabilities, the resource pool scheduling strategy configuration module in the container platform-based GPU resource pool dynamic management system supports a series of dynamically configurable resource allocation scheduling strategies to meet the differentiated requirements of different business scenarios. The most basic is the standard resource allocation scheduling strategy, and users only need to specify the required GPU resource quantity in the format of {"GPU quantity":"x"}, and the system will automatically select available GPU resources for allocation according to the default scheduling mechanism. For scenarios that require more refined control, the module provides more rich policy options. For example, users can use the node label-based resource allocation scheduling strategy to specify that the system allocates corresponding GPU resources from nodes that meet specific label conditions by using the format {"GPU quantity":"x","GPU scheduling strategy":{"node label":"a certain node label"}}. Similarly, users can also submit GPU type-based resource allocation scheduling strategies in the format {"GPU quantity":"x","GPU scheduling strategy":{"GPU type":"a certain GPU type"}}, which is used to specify GPUs of a specific model. For scenarios that require high-performance communication, the module also supports GPU topology connection type-based resource scheduling strategies in the format {"GPU quantity":"x","GPU scheduling strategy":{"GPU topology connection type":"a certain connection type"}}, which is used to allocate GPU resources with a specific interconnection method. As a concentrated embodiment of its flexibility and powerful functions, the module also supports mixed scheduling strategies, allowing users to combine multiple scheduling strategies, such as {"GPU quantity":"x","GPU scheduling strategy":{"node label":"a certain node label","GPU type":"a certain GPU model","GPU topology connection type":"a certain connection type"}}, to filter and allocate GPU resources that meet all strategy requirements, thereby achieving the highest degree of scheduling flexibility and accuracy.

[0055] Specifically, the resource pool scheduling coordination control module is the core component of the on-demand dynamic management and intelligent scheduling of the GPU resource pool. It is connected with the resource pool scheduling strategy configuration module and is responsible for resource matching and scheduling decisions based on the resource requests transmitted by the module and the preset cluster resource state, and finally generates instructions for the dynamic management of the GPU resource pool. Compared with the existing technology that generally adopts a static resource pool structure and fixed scheduling logic, this module innovatively introduces a resource pool dynamic creation mechanism based on scheduling strategy driving and supports real-time response to user resource requests and scheduling strategies. The process of the resource pool dynamic creation mechanism based on scheduling strategy driving is as shown in Figure 5

[0056] ​The core function of the module is that it can trigger the creation, update or release operation of the GPU resource pool according to the user's resource request and the configured scheduling strategy. This ability fundamentally breaks the separation between resource scheduling and resource management in traditional systems, and builds a highly linked scheduling closed loop from demand perception to resource change, thereby realizing the dynamic matching of resource supply and task demand. In order to make accurate scheduling decisions, the module is also responsible for synchronously gathering key metadata information in the cluster, including but not limited to node state, GPU resource attributes, current allocation state, and resource pools to which each GPU belongs. Based on these real-time and global resource information, the module makes intelligent scheduling decisions. By deeply integrating global resource state perception capability and scheduling execution mechanism, the GPU resource pool dynamic management system based on container platform not only can significantly improve the overall utilization of GPU resources, but also greatly enhances the efficiency and flexibility of task scheduling, ensuring the smoothness and efficiency of the entire dynamic management process.

[0057] Specifically, the resource pool dynamic management module is connected with the resource pool scheduling coordination control module, and as the system execution core, it is deployed on each physical node of the cluster using a distributed architecture. The core function of the module is to execute the dynamic management instructions issued by the coordination control module to manage one or more GPU resources in a pool, thereby dynamically creating, updating or releasing GPU resource pools, and reporting the GPU resource state after the operation is completed to update the overall resource state of the cluster.

[0058] Specifically, the module is responsible for automatically discovering, real-time state monitoring and health assessment of the GPU hardware resources in each node, and continuously updates the state of the node, the GPU attributes, the allocation of the GPU and the information of the resource pool to which the GPU belongs, thereby ensuring that the system can accurately grasp the real-time running situation of all GPU resources. In addition, the resource pool dynamic management module has high responsiveness, and it can immediately process various resource operation requests including creating, updating and releasing GPU resource pools by listening to task instructions from the resource pool scheduling coordination control module.

[0059] To avoid the need for user services to explicitly perceive the differences between heterogeneous GPU resources, and to ensure that GPU resource pools created by different users can be isolated from each other, the module introduces an innovative resource pool naming mechanism. The format of the mechanism is: <uniform resource pool domain> / <user-defined namespace>. Among them, <uniform resource pool domain> is a standardized field predefined by the system, and all user-created resource pools belong to this uniform domain, which ensures the uniform matching of the resource scheduling strategy. And <user-defined namespace> is defined based on the user's unique identifier, used to achieve resource isolation and personalized naming management between different users or business units, supporting naming isolation and flexible expansion in a multi-user environment. By dividing the resource pool naming into two levels, the system realizes the decoupling of resource type abstraction and user identity identification. This hierarchical naming mechanism not only successfully masks the differences between underlying heterogeneous GPU resources, but also significantly improves the manageability and intelligent level of scheduling of the system.

[0060] The embodiment of the application also provides a GPU resource pool dynamic management process, which includes a GPU resource pool dynamic creation process, a GPU resource pool dynamic expansion process, a GPU resource pool dynamic contraction process, and a GPU resource pool dynamic release process. Specifically, as shown in Figure 6 The dynamic creation process of the GPU resource pool starts with a resource request from the user, and through the cooperative work of each module, finally realizes the generation and management of a brand new GPU resource pool that accurately matches the user's needs. First, the user submits a resource request containing the required number of GPUs and a specific scheduling policy through the resource pool scheduling policy configuration module. After receiving the request, the resource pool scheduling coordination control module will immediately query the current cluster resource state to filter out all GPU resources that meet the user's scheduling policy and are in an idle state. After completing resource matching and forming a scheduling decision, the coordination control module will issue a creation instruction to the resource pool dynamic management module deployed on one or more target physical nodes. After receiving the instruction, the dynamic management module on the node will perform specific pooling operations, such as adding a uniform resource pool tag to the specified GPU resources. After the operation is completed, each dynamic management module will report the execution result to the coordination control module, which will update the global cluster resource state after confirming the success of the operation, and formally include the newly created resource pool and its contained GPU information into system management, thus completing the closed-loop process of the entire dynamic creation.

[0061] The dynamic expansion process of the GPU resource pool aims to add new GPU resources to the existing resource pool as needed to meet the growing business load demand. As shown in Figure 7As shown, the process starts with a user initiating an expansion request, explicitly indicating the target resource pool that needs to be expanded, as well as the number of GPUs to be added and the corresponding scheduling policy. After receiving the request, the resource pool scheduling coordination control module will search and match the idle GPU resources in the cluster according to the scheduling policy in the expansion request. Once the eligible GPU resources are found, the coordination control module will generate an expansion decision and issue instructions to the physical nodes where these new resources are located. The resource pool dynamic management module on the nodes executes the instructions, adding these new GPU resources to the target resource pool, usually by tagging them with the same label as the target resource pool. After successful execution and reporting of the results, the resource pool scheduling coordination control module updates the cluster resource state, officially registering the newly added GPUs under the name of the target resource pool, thus completing the online dynamic expansion of the resource pool.

[0062] The dynamic contraction process of the GPU resource pool is used to safely remove part of the GPU resources from an existing resource pool to release idle computing power and optimize resource utilization. As shown, Figure 8 The process starts with a user submitting a contraction request, which specifies the target resource pool and the number of GPUs to be removed. After receiving the request, the resource pool scheduling coordination control module analyzes the current usage status of each GPU in the target resource pool and selects a specified number of GPUs that are idle or can be safely recycled as contraction objects. After determining the list of GPUs to be removed, the coordination control module issues removal instructions to the nodes where these GPUs are located. The resource pool dynamic management module on each node removes the association of these GPUs with the original resource pool, such as removing their resource pool tags, when executing the instructions. After the operation is completed, the execution results are reported to the coordination control module, which immediately updates the cluster resource state, marking the removed GPUs as idle and removing them from the member list of the original resource pool, thus achieving smooth contraction of the resource pool.

[0063] The dynamic release process of the GPU resource pool is used to completely remove and release a GPU resource pool that is no longer needed and all GPU resources it contains from the system. As shown, Figure 9As shown, the process is initiated by a user submitting a release request, in which only the name of the target resource pool to be released needs to be specified. After receiving the request, the resource pool scheduling coordination control module queries the cluster resource state to identify all GPU members belonging to the resource pool. Subsequently, it issues release instructions to all relevant nodes containing these GPUs. After receiving the instructions, the resource pool dynamic management module on each node performs resource unbinding operations, i.e., removes all labels or identifiers associated with the resource pool on the target GPU. When all nodes successfully perform the operations and report the results, the resource pool scheduling coordination control module performs the final state synchronization, updates all released GPUs to the idle state in the cluster resource state, and completely deletes the definition of the resource pool, thereby safely returning all computing resources to the global available resource pool

[0064] The embodiment of the present application also provides a computer readable storage medium, which stores computer program codes, and when the computer program codes run on a computer, the computer is caused to execute the related method steps to realize the GPU resource pool dynamic management method based on a container platform provided by the embodiment.

[0065] The embodiment of the present application also provides a computer program product, which, when running on a computer, causes the computer to execute the related steps to realize the GPU resource pool dynamic management method based on a container platform provided by the embodiment.

[0066] Among them, the system, computer readable storage medium or computer program product provided by the embodiment of the present application are used to execute the corresponding method provided by the above, so the beneficial effects that can be achieved are referred to the beneficial effects in the corresponding method provided by the above, which will not be described here.

[0067] It should be noted that: the above-mentioned sequence of the embodiments is only for description, not representing the advantages and disadvantages of the embodiments.

[0068] The processes depicted in the figures do not necessarily require the particular order shown, or sequential order, to achieve the desired results. In some implementations, multitasking and parallel processing can be advantageous. The various embodiments described in this specification are described using a progression of embodiments, each described embodiment can be combined with other embodiments in this specification to describe different variations that are not necessarily mutually exclusive.

Claims

1. A method for dynamic management of GPU resource pools based on a container platform, characterized in that, Includes the following steps: The user's resource request is parsed to obtain the parsed scheduling requirements; Based on the parsed scheduling requirements and cluster resource status, resource matching and scheduling decisions are made to obtain the scheduling decision; Based on scheduling decisions, resource pool management instructions are issued and executed to obtain operational results; Based on the operation results, cluster status aggregation and metadata synchronization are performed to obtain the updated cluster resource status.

2. The method for dynamic management of GPU resource pools based on a container platform according to claim 1, characterized in that, The user's resource request is parsed to obtain the parsed scheduling requirements, including: The user resource request is parsed and its validity is verified to obtain the verified user resource request; Extract core requirement parameters from verified user resource requests to obtain the number of GPUs extracted; Extract the GPU scheduling policy from the verified user resource request, and perform scheduling policy-filter mapping and generation on the GPU scheduling policy to obtain a filter condition list; The extracted number of GPUs and the list of filter conditions are aggregated to obtain the parsed scheduling requirements.

3. The method for dynamic management of GPU resource pools based on a container platform according to claim 2, characterized in that, Based on the parsed scheduling requirements and cluster resource status, resource matching and scheduling decisions are performed to obtain the scheduling decision, including: Based on the list of filter conditions in the parsed scheduling requirements, candidate GPU resources are identified in the cluster resource status to obtain a candidate GPU list. Based on the number of GPUs extracted from the parsed scheduling requirements, the resource sufficiency of the candidate GPU list is determined to obtain a sufficiency flag. In response to the sufficiency flag being true, the candidate GPU list is scored and sorted based on the allocation strategy and the number of extracted GPUs to obtain a sorted list of schemes. Based on the request context, the optimal GPU set is selected and the resource pool is named from the sorted list of schemes to obtain the scheduling decision.

4. The method for dynamic management of GPU resource pools based on a container platform according to claim 3, characterized in that, Based on the request context, the optimal GPU set is selected and the resource pool is named from the sorted list of solutions to obtain the scheduling decision, including: Based on the request context, the optimal GPU set is selected and the resource pool is named from the sorted list of solutions to obtain the selected GPU list and the generated resource pool name; The scheduling decision is obtained by aggregating the selected GPU list and the generated resource pool names.

5. The method for dynamic management of GPU resource pools based on a container platform according to claim 3, characterized in that, In response to the sufficiency flag being true, the candidate GPU list is scored and sorted based on the allocation strategy and the number of extracted GPUs to obtain a sorted list of schemes, including; Input the number of extracted GPUs and the list of candidate GPUs into the combinatorial generator; The combinatorial generator applies combinatorial mathematical algorithms to process the extracted number of GPUs and the list of candidate GPUs to obtain a list of possible solutions; Based on the allocation strategy, each possible solution in the list of possible solutions is scored and structured to obtain an unsorted list of scored solutions. The unsorted list of scored solutions is sorted by solution priority to obtain a sorted list of solutions.

6. The method for dynamic management of GPU resource pools based on a container platform according to claim 5, characterized in that, The allocation strategy includes a packing strategy and a distribution strategy.

7. The method for dynamic management of GPU resource pools based on a container platform according to claim 6, characterized in that, Based on the allocation strategy, each possible solution in the list of possible solutions is scored and structured to obtain an unsorted list of scored solutions, including: Extract the first possible solution from the list of possible solutions; Extract the number of unique node IDs from the first possible solution; If the allocation strategy is a packaging strategy, the score of the first possible solution is 1 divided by the number of unique node IDs; If the allocation strategy is a distributed strategy, the score of the first possible solution is the number of unique node IDs.

8. A container-based GPU resource pool dynamic management system, used to execute the container-based GPU resource pool dynamic management method according to any one of claims 1-7, characterized in that, include: The resource pool scheduling strategy configuration module is used to receive user-defined resource requests that include GPU resource requirements and scheduling strategies. The resource pool scheduling coordination and control module is connected to the resource pool scheduling strategy configuration module. It is used to perform resource matching and scheduling decisions based on the resource request and the preset cluster resource status, and generate instructions for dynamic management of the GPU resource pool. The resource pool dynamic management module is connected to the resource pool scheduling and coordination control module. It is used to execute the dynamic management instructions to pool one or more GPU resources, thereby dynamically creating, updating or releasing GPU resource pools, and reporting the GPU resource status to update the cluster resource status.

Images