A resource scheduling method and device, electronic equipment and storage medium

Abstract

This invention discloses a resource scheduling method, apparatus, electronic device, and storage medium, relating to the field of computer technology. The resource scheduling method includes: acquiring a service request; determining the service type, target resource group, and node resource status of the target resource group corresponding to the service request; determining resource scheduling parameters corresponding to the service request based on the service type; and determining target node resources based on the node resource status and the resource scheduling parameters; wherein the resource scheduling parameters include the target node operating mode and resource specification parameters; and executing the service request according to the target node resources. This invention achieves resource isolation for different services, avoids resource contention and interference, and supports different node operating modes, breaking the limitation of existing technologies that only support dedicated GPU resources, improving GPU resource utilization, and reducing resource idleness caused by small-scale services.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a resource scheduling method, apparatus, electronic device, and storage medium. Background Technology

[0002] In existing technologies, cluster resource isolation and scheduling are mostly limited to the resource pool level or node level. For graphics processing unit (GPU) resources, it is usually only possible to perform coarse-grained partitioning based on node model, number of nodes, or tenant resource groups. It is difficult to continue to achieve finer-grained GPU reuse and partitioning within dedicated resource groups that have already achieved tenant isolation.

[0003] However, existing technologies typically only support exclusive allocation of entire GPUs or entire nodes, making it difficult to achieve parallel reuse of multiple tasks within the same physical node. Large-scale tasks may not be scheduled in a timely manner because nodes with a high number of GPUs are occupied by scattered small tasks, resulting in low overall resource utilization. Therefore, how to achieve fine-grained scheduling of GPU-level sharing and memory-level partitioning to meet the cluster computing power scheduling requirements of high utilization, high controllability, and strong isolation has become an urgent problem to be solved. Summary of the Invention

[0004] This invention provides a resource scheduling method, apparatus, electronic device, and storage medium to solve the resource scheduling problem in the prior art that cannot satisfy the coexistence of high utilization, high controllability, and strong isolation.

[0005] According to one aspect of the present invention, a resource scheduling method is provided, wherein the method includes: Obtain the business request, determine the business type, target resource group, and node resource status of the target resource group corresponding to the business request; Based on the business type, the resource scheduling parameters corresponding to the business request are determined, and the target node resources are determined according to the node resource status and the resource scheduling parameters; wherein, the resource scheduling parameters include the target node operating mode and resource specification parameters; The service request is executed based on the resources of the target node.

[0006] According to another aspect of the present invention, a resource scheduling apparatus is provided, wherein the apparatus comprises: The information acquisition module is used to acquire business requests, determine the business type, target resource group, and node resource status of the target resource group corresponding to the business request; The resource determination module is used to determine the resource scheduling parameters corresponding to the service request based on the service type, and to determine the target node resources according to the node resource status and the resource scheduling parameters; wherein, the resource scheduling parameters include the target node operating mode and resource specification parameters; The resource scheduling module is used to execute the business request based on the resources of the target node.

[0007] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the resource scheduling method according to any embodiment of the present invention.

[0008] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the resource scheduling method described in any embodiment of the present invention.

[0009] According to another aspect of the present invention, embodiments of the present invention also provide a computer program product, the computer program product including a computer program, which, when executed by a processor, implements the resource scheduling method of any embodiment of the present invention.

[0010] The technical solution of this invention obtains a service request, determines the service type, target resource group, and node resource status of the target resource group corresponding to the service request, determines the resource scheduling parameters corresponding to the service request based on the service type, wherein the resource scheduling parameters include the target node operating mode and resource specification parameters, determines the target node resources according to the node resource status and resource scheduling parameters, and executes the service request according to the target node resources, thereby achieving resource isolation for different services, avoiding resource contention and interference, and supporting different node operating modes. This breaks the limitation of existing technologies that only support dedicated use of the entire card, improves the utilization rate of GPU resources, and reduces resource idleness caused by small-scale services.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart of a resource scheduling method provided in Embodiment 1 of the present invention; Figure 2 This is a flowchart of a resource scheduling method provided according to Embodiment 2 of the present invention; Figure 3 This is a flowchart of a resource scheduling method provided according to Embodiment 3 of the present invention; Figure 4 This is a flowchart of a resource group construction method provided in Embodiment 3 of the present invention; Figure 5 This is a flowchart of a shared GPU method provided according to Embodiment 3 of the present invention; Figure 6 This is a flowchart of a video memory partitioning method provided in Embodiment 3 of the present invention; Figure 7 This is a schematic diagram of a resource group construction device according to Embodiment 4 of the present invention; Figure 8 This is a schematic diagram of the structure of an electronic device that implements the resource group construction method of the present invention. Detailed Implementation

[0014] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0015] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0016] Example 1 Figure 1This is a flowchart of a resource scheduling method according to Embodiment 1 of the present invention. This embodiment is applicable to the allocation and scheduling of resources for cluster nodes. The method can be executed by a resource scheduling device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the method includes: S110. Obtain the business request, determine the business type, target resource group, and node resource status of the target resource group corresponding to the business request.

[0017] In this context, a business request can be understood as a task requirement that occupies GPU resources. For example, a business request may include, but is not limited to, model inference and data processing. Generally, a business request can carry information such as the business type and resource requirements. Business requests can be submitted by a business system. The business type refers to the specific attributes of the business; for example, a business type may include inference tasks, training tasks, etc. In actual operation, the business type may be related to the specifications and operating mode of the required GPU resources. A target resource group refers to a dedicated set of resources used to isolate resources from different businesses or tenants, ensuring the independence of resource allocation and avoiding mutual interference. Node resource status refers to the resource usage of nodes in the cluster. For example, node resource status may include information such as node operating mode and remaining resource quantity.

[0018] In this embodiment, a service request can be received, the service type corresponding to the request can be determined, and the resource group to which the request belongs can be identified as the target resource group. Information about all nodes within the target resource group is extracted, including whether the node is currently idle, its remaining GPU memory, remaining computing power, and its node operating mode. Generally, node operating modes can include exclusive mode, shared mode, and GPU memory partitioning mode. Simultaneously, the presence of running tasks on the node is recorded. In actual operation, different service requests can be pre-set with corresponding target resource groups, and the node operating modes of different nodes within the target resource group can be different.

[0019] S120. Determine the resource scheduling parameters corresponding to the business request based on the business type, and determine the target node resources based on the node resource status and resource scheduling parameters.

[0020] Resource scheduling parameters refer to the parameters used to allocate node resources, which are the resource parameters required by the business request. In practical applications, resource scheduling parameters may include information such as the Central Processing Unit (CPU), memory, number of GPUs, GPU memory, GPU list, and target node operating mode. Specifically, resource scheduling parameters include the target node operating mode and resource specification parameters. The target node operating mode refers to the operating mode that the target GPU node must possess, determined according to the business type of the current business request. Generally, the target node operating mode corresponding to each business type can be pre-set. In one embodiment, the target node operating mode may include exclusive mode, shared mode, and memory partitioning mode. Resource specification parameters are the quantitative configuration indicators of hardware resources required by the business request matched according to the business type, such as CPU, memory, number of GPUs, GPU memory, and GPU list. Generally, different business types correspond to different resource specification parameters. Target node resources refer to the specific resource configuration that meets the business requirements. For example, target node resources may include, but are not limited to, information such as GPU computing power and memory.

[0021] In this embodiment, pre-set target node operating modes and resource specification parameters associated with the business type can be extracted, and these parameters can be used as resource scheduling parameters corresponding to the business request. In actual operation, parameters such as the number of GPUs, memory size, and CPU can be matched according to the business type, and the target node operating mode required by the business type can be determined. These parameters, along with the target node operating mode, are then used as resource scheduling parameters corresponding to the business request. Idle nodes in the target resource group are then determined based on the node resource status, and node resources that meet the resource scheduling parameters are matched among these idle nodes as target node resources. Alternatively, in actual operation, the target node operating mode corresponding to the business type can be determined first, and a pre-set resource mapping template can be extracted. The number of GPUs and memory required by the business request can be generated according to the resource mapping template. Nodes with the target node operating mode are identified in the node resource status as nodes to be allocated, and nodes that meet the GPU number and memory specifications among the nodes to be allocated are identified as target nodes. When the target node is running in exclusive mode, all resources of the target node can be used as the target node resources. When the target node is running in shared mode or memory partitioning mode, the resources required for the service request can be allocated according to the resource specification parameters on the target node. In one embodiment, if the target node is running in memory partitioning mode, it is necessary to additionally verify whether the node supports virtual GPU to ensure that memory partitioning allocation can be performed.

[0022] S130, Execute the business request based on the target node resources.

[0023] In one embodiment, a container runtime environment can be created based on the target node resources. Business tasks can be deployed into the container, and corresponding resource configurations, such as vGPU resource parameters and environment variables, can be injected to ensure that the tasks only use the allocated GPU resources. In another embodiment, during the execution of business requests, the container's running status can be monitored in real time to ensure that business tasks only use the allocated resources and avoid resource contention.

[0024] In this embodiment of the invention, by acquiring a service request, the service type, target resource group, and node resource status of the target resource group corresponding to the service request are determined. Based on the service type, resource scheduling parameters corresponding to the service request are determined. Based on the node resource status and resource scheduling parameters, target node resources are determined. The service request is executed based on the target node resources, thereby achieving resource isolation for different services, avoiding resource contention and interference, and supporting different node operating modes. This breaks the limitation of existing technologies that only support dedicated use of the entire GPU, improves the utilization rate of GPU resources, and reduces resource idleness caused by small-scale services.

[0025] In one embodiment, before acquiring the service request, determining the service type, target resource group, and node resource status of the target resource group corresponding to the service request, the method further includes: Select nodes in the cluster that match the target resource specifications, and build resource groups based on the nodes; Determine the node operating mode for each node; the node operating mode includes exclusive mode, shared mode, and video memory partitioning mode. Node type labels are created for nodes according to their operating modes, and resource group labels are created for each node.

[0026] The target resource specification refers to the preset requirements of a business or tenant for the node's hardware resources, including the number of CPU cores, GPUs, video memory size, and memory capacity, to facilitate the division of different resource groups. In actual operation, the target resource specification can be customized according to business needs. A resource group is a dedicated set of resources composed of several nodes that meet the target resource specification, used to isolate resources from different businesses and tenants, ensuring the independence of resource allocation. Exclusive mode means that all GPU, CPU, and other resources of a node are allocated to only one business task, so that the task exclusively uses the entire node's resources, with no resource sharing; shared mode means that the node's GPU resources can be shared by multiple business tasks, with multiple tasks occupying the same node's GPU computing power or video memory in parallel, suitable for small-scale, low-interference businesses; video memory partitioning mode means that the node's physical GPU video memory is partitioned into multiple virtual graphics processing units (vGPUs), with each task only occupying a portion of the video memory and computing power, realizing the reuse of a single physical GPU by multiple tasks. The node type label is a marker used to identify the node's operating mode, facilitating quick identification of the node's operating mode. Resource group tags are markers used to identify the resource group to which a node belongs. The tag value is a unique identifier for the resource group, making it easier to filter nodes within the resource group.

[0027] In this embodiment, nodes that meet the target resource specifications can be selected and added to the resource group as dedicated nodes. After joining, a resource group tag and a node-level tag are written to the node, and taints are configured so that subsequent services can only be scheduled to nodes within the corresponding resource group. In one embodiment, the resource group tag is used to identify which dedicated resource group the node belongs to, and the resource group node tag is used to identify the unique node identity of the node within the resource group. Through these two levels of tags, a two-layer resource organization structure of resource group-level queues and node-level queues can be formed, restricting tasks to only enter the target tenant's resource group and allowing tasks to be further directed to specific nodes within that resource group.

[0028] In one embodiment, the resource scheduling method further includes: If there is no node in the target resource group that operates in the target node's operating mode, extract the idle nodes in the target resource group as candidate nodes; Identify candidate nodes that support the switching of operating modes as target switching nodes, and switch the operating mode of the target switching nodes to the target node's operating mode.

[0029] Idle nodes refer to nodes within the target resource group that currently have no running business tasks and whose resources are in an idle state.

[0030] In this embodiment, when it is determined that there is no node in the target resource group whose running mode is the target node's running mode, the current task status of the node can be queried to identify idle nodes in the target resource group, and these idle nodes can be used as candidate nodes. If the target node's running mode is a memory partitioning mode, it can be verified whether the node supports vGPU and whether the corresponding virtualization driver is installed; if the target node's running mode is a shared mode, it can be verified whether the node's GPU supports multi-task sharing. If these conditions are met, the running mode of the target switching node can be switched to the target node's running mode. In actual operation, when the target node's running mode is a memory partitioning mode or a shared mode, the original tag value can be backed up to a preset backup tag, the current node type tag can be rewritten, and the tag can be adjusted back to the original tag after the business request is executed. This allows the node to be dynamically switched to the running mode required by the business, improving the scheduling success rate and resource utilization without adding new hardware.

[0031] Example 2 Figure 2 This is a flowchart of a resource scheduling method according to Embodiment 2 of the present invention. This embodiment is a further optimization and extension based on the above embodiments, and can be combined with various optional technical solutions in the above embodiments. Figure 2 As shown, the method includes: S210. Receive a service request and identify the service type and target resource group corresponding to the service request.

[0032] In this embodiment, a pre-defined system can receive service requests submitted by the system, determine the service type by parsing the task description and service identifier in the service request, and match the corresponding target resource group based on the service identifier. Generally, the target resource groups corresponding to different service requests can be pre-set. For example, service requests from the same tenant can be assigned to the same target resource group.

[0033] S220. Extract the current remaining resources and node operating mode of the nodes within the target resource group, and use the current remaining resources and node operating mode as the node resource status.

[0034] The remaining resources can be understood as the unused hardware resources of each node in the target resource group. The remaining resources can include information such as the number of CPU cores remaining, the number of GPUs remaining, the amount of video memory remaining, and the amount of RAM remaining.

[0035] In this embodiment, all nodes within the target resource group can be traversed to obtain the real-time status information of each node, collect the real-time resource usage of each node, and calculate the current remaining resources. In one embodiment, the remaining CPU, GPU, video memory, and RAM of each node can be determined as the current remaining resources, and the node operating mode of each node can be determined. Generally, the node type tag of each node can be queried to obtain the current operating mode of the node; this tag is configured when the resource group is built. The current remaining resources and the node operating mode are used as the node resource status.

[0036] S230. Extract the preset resource mapping template, match the target node operating mode and resource specification parameters corresponding to the business type in the preset resource mapping template, and use the target node operating mode and resource specification parameters as resource scheduling parameters.

[0037] Among them, the preset resource mapping template refers to the pre-configured template of the correspondence between business types and resource requirements. It can record the target node operation mode and resource specification parameters corresponding to different business types, and realize the standardized matching of scheduling parameters.

[0038] In this embodiment, a preset resource mapping template can be retrieved, and the corresponding target node operating mode and resource specification parameters can be found in the preset resource mapping template according to the business type. The extracted target node operating mode and resource specification parameters (such as 1 vGPU and 8GB video memory) are integrated as resource scheduling parameters for the business request.

[0039] S240. Extract nodes whose node operating mode is the target node operating mode from the node resource status and use them as nodes to be allocated.

[0040] In this embodiment, nodes whose operating modes are completely consistent with the target node's operating mode can be selected as nodes to be assigned.

[0041] S250. Determine the node to be allocated that meets the resource specification parameters as the target node, and use the target allocation resources in the target node as the target node resources.

[0042] Among them, resource specification parameters refer to the specific resource quantification requirements required for the normal operation of the business, including parameters such as the number of CPU cores, the number of GPUs (or the number of vGPUs), the size of video memory (or the video memory quota of a single vGPU), and the memory capacity.

[0043] In this embodiment, the nodes to be allocated can be traversed, and the current remaining resources of each node can be compared with the resource specification parameters to filter out nodes whose remaining resources fully meet the resource specification parameters. For example, when the resource specification parameters are 1 vGPU, 8GB of video memory, and 8-core CPU, nodes with ≥8GB of remaining video memory, ≥8 cores of remaining CPU, and support for vGPU partitioning can be filtered out as target nodes. When there are multiple target nodes, the optimal node, such as the node with the most abundant remaining resources and the lowest load, can be selected as the target node for the business request; alternatively, target nodes can be selected according to a preset strategy (such as load balancing).

[0044] In one embodiment, when the target node's operating mode is memory partitioning mode, the target allocated resources in the target node are used as the target node resources, including: Determine the available resources of the target node, and match the resources with the same resource specification parameters as the target node resources.

[0045] In this embodiment, the real-time resource status of the target node can be queried, such as the current remaining video memory size and the minimum video memory quota per vGPU, to determine the number of vGPUs that can be allocated to the target node and the video memory range of a single vGPU. Among the available video memory of the target node, resources that completely match the resource specification parameters are matched. The successfully matched vGPU resources are then combined with the required CPU and memory resources and integrated as the target node resources.

[0046] S260. Create a corresponding container runtime environment based on the target node resources, and deploy the business tasks corresponding to the business requests to the container for execution.

[0047] The container runtime environment refers to an independent and isolated runtime environment built on the target node resources, such as Docker containers, which can ensure that business tasks only use the allocated target node resources and do not interfere with other tasks or resources on the node.

[0048] In this embodiment, a container creation command can be sent to the target node. Based on the target node's resources, the container's runtime environment is configured, and the maximum resource limit that the container can use is completely consistent with the target node's resources. Corresponding resource configuration environment variables are injected to ensure that the container can only access the allocated target node resources, does not occupy other idle resources on the node, and does not interfere with other tasks. The business task corresponding to the business request is then uploaded to the created container for execution.

[0049] In this embodiment of the invention, by receiving a service request, identifying the service type and target resource group corresponding to the service request, extracting the current remaining resources and node operating modes of the nodes within the target resource group, using the current remaining resources and node operating modes as the node resource status, extracting a preset resource mapping template, matching the target node operating mode and resource specification parameters corresponding to the service type in the preset resource mapping template, using the target node operating mode and resource specification parameters as resource scheduling parameters, extracting nodes whose node operating mode is the target node operating mode from the node resource status as nodes to be allocated, determining nodes to be allocated that meet the resource specification parameters as target nodes, using the target allocated resources in the target nodes as target node resources, creating a corresponding container runtime environment based on the target node resources, and deploying the service task corresponding to the service request to run in the container, this achieves fine-grained allocation of GPUs, avoids resource waste caused by full card occupancy, and significantly improves GPU resource utilization.

[0050] Example 3 Figure 3 This is a flowchart of a resource scheduling method according to Embodiment 3 of the present invention. This embodiment, based on the above embodiments, uses a dedicated GPU mode as the dedicated mode and a shared GPU mode as the shared mode, and a dedicated resource group node queue as the resource group, to further illustrate a resource scheduling method. Figure 3 As shown, the method includes: This invention presents a complete closed loop encompassing resource group creation, adding physical nodes to the resource group node queue, node mode settings (including dedicated GPU mode, shared GPU mode, and memory partitioning mode), resource template creation, business task submission, generation of resource group constraints, node constraints, and mode constraints, submission to K8 server scheduling, container runtime injection, and monitoring write-back and state recovery. Specifically, the dedicated resource group queue addresses tenant isolation, node mode settings determine the GPU management method used within the resource group, template definitions abstract business specifications into schedulable objects, and monitoring write-back and state recovery ensure system consistency. The method generally includes the following steps: 1. Establish a dedicated resource group and include physical nodes that meet the conditions into the dedicated resource group node queue.

[0051] 2. Set the running mode for the nodes in the dedicated resource group. The running mode includes at least: dedicated GPU mode, shared GPU mode, and video memory partitioning mode.

[0052] 3. Define the required specifications for the business through resource templates. The templates shall include at least the following fields: CPU, memory, number of GPUs, GPU video memory, GPU list, and node type.

[0053] 4. Receive business task requests and generate scheduling constraints based on resource groups, node modes, and template parameters.

[0054] 5. For shared GPU tasks, the task is constrained to a specified GPU card within the node based on the list of GPU card numbers.

[0055] 6. For the video memory allocation task, generate a vGPU video memory resource request based on the video memory value.

[0056] 7. Submit the generated resource request to the K8s scheduler for execution, and inject the corresponding environment variables into the container at runtime.

[0057] 8. Monitor task running status, node resource status, and resource group statistics, and execute recovery logic when a node is deleted, a mode is canceled, or a resource group is migrated.

[0058] In one embodiment, Figure 4 This is a flowchart of a resource group construction method according to Embodiment 3 of the present invention. Figure 4 This illustrates the process of a node moving from a public physical node pool to a dedicated resource group queue. By tagging and tainting nodes, a dedicated binding between a node and a resource group is achieved. This binding not only restricts the visibility to tenants but also provides a unified underlying node for subsequent shared GPUs and video memory allocation.

[0059] The construction of the dedicated resource group node queue is as follows: First, the system creates a resource group object and selects nodes that meet the conditions of CPU, memory, GPU model, and number of GPUs to be added to the resource group as dedicated nodes. After being added, the system writes resource group tags and node-level tags to the nodes and configures taints so that subsequent services can only be scheduled to nodes within the corresponding resource group.

[0060] In one embodiment, the following constraint fields may be used: resource group label: aicp.group / resource_group; resource group node label: aicp.group / resource_group_node.

[0061] Resource group tags identify which dedicated resource group a node belongs to; resource group node tags identify the unique identity of that node within the resource group. These two levels of tags create a two-tiered resource organization structure: a "resource group-level queue + node-level queue." The first tier restricts tasks to the target tenant's resource group; the second tier allows tasks to be further directed to specific nodes within that resource group.

[0062] In one embodiment, Figure 5 This is a flowchart of a shared GPU method provided in Embodiment 3 of the present invention. Figure 5The implementation details of shared GPUs are shown. The key is not simply allowing multiple tasks to share a single node, but rather achieving card-level binding through a GPU card index list. This ensures that each task uses only a designated physical GPU card, avoiding conflicts between tasks on the same card. This mechanism is a second-level fine-grained scheduling performed after isolation by dedicated resource groups.

[0063] Specifically, the shared GPU functionality refers to setting a node within a dedicated resource group as a shareable GPU node, allowing different tasks to bind different physical GPU cards and run in parallel on the same node. The implementation process includes: receiving a shared mode setting request and marking the target node as a shared GPU node. If the target node does not support sharing, the setting is rejected. If a shared template that depends on the GPU list still exists when canceling sharing, cancellation is rejected. A template containing a GPU list (gpu_list) is created, and gpu_list is written into the template, representing the set of physical GPU card indices that are allowed to be used. When a business task is created, if a custom GPU card number list (custom_gpu_list) is included, the target node is verified to have been set as a shared GPU node. The resource construction module converts custom_gpu_list into the gpu_list attribute. The container runtime injects the NVIDIA visible device list (0NVIDIA_VISIBLE_DEVICES) environment variable, making the container only visible to the allocated physical GPU cards, and tasks run in isolation by card within the dedicated node. In this mode, the system no longer requests GPU resources in the form of a whole card, but submits tasks with the minimum CPU and memory requests, thereby supporting card-level parallel reuse on the same node.

[0064] In one embodiment, Figure 6 This is a flowchart of a video memory partitioning method provided in Embodiment 3 of the present invention. Figure 6 The process of switching and scheduling video memory partitioning modes is illustrated. Unlike existing node-level resource grouping schemes, this invention does not simply statically group 4-card nodes and 8-card nodes, but rather further constructs independent partitioning resource requests based on video memory values ​​within the already grouped dedicated nodes, thereby achieving video memory-level resource scheduling.

[0065] Specifically, the video memory allocation function refers to switching a node in a dedicated resource group to a video memory allocation node, enabling business tasks to request GPU resources based on a share of video memory rather than the entire card. Its implementation process includes: Upon receiving a request to set the video memory partitioning mode, the system switches the node to a video memory partitioning node, verifying that the target node supports virtualized GPUs; it also verifies that there are no running resources on the node during the switch; it reads the current node type label aicp.group / aipods_type, and if switching to video memory partitioning mode, it first backs up the original label value to a preset backup label, such as pre.aicp.group / aipods_type, and then rewrites the current node type label to vGPU. A video memory partitioning template is created, which at least includes the number of CPUs (gpu_count) and the CPU memory capacity (gpu_memory). The resource construction module generates video memory partitioning resource requests based on gpu_count and gpu_memory, and schedules them according to video memory share within a dedicated resource group. In one embodiment, for NVIDIA, qingcloud.nvidia.com / vgpu and qingcloud.nvidia.com / vgpumem can be generated, and resource group constraints, node constraints, and aipods_type=vGPU constraints are attached during scheduling. The original label is restored when the partitioning is canceled. For example, if the original label is dedicated GPU mode, the label can be changed to dedicated GPU mode after the task is completed.

[0066] In one embodiment, this embodiment also provides a corresponding system, which includes at least: a resource group creation module for creating dedicated resource groups and maintaining the relationship between tenants and resource groups; a node management module for adding physical nodes to the dedicated resource group node queue; a sharing mode control module for switching nodes to shared GPU nodes or canceling sharing; a sharding mode control module for switching nodes to memory sharding nodes or canceling sharding; a template definition module for generating exclusive, shared, and sharded templates; a resource translation module for converting template parameters into K8s resource requests, tag constraints, and runtime injection information; a scheduling execution module for submitting Pods and triggering K8s scheduling; and a status write-back module for writing back node status, resource group statistics, and monitoring results.

[0067] This embodiment introduces a shared GPU node mode and a memory-segmented node mode based on the dedicated resource group node partitioning queue. A unified resource template mechanism simultaneously expresses three specifications: dedicated GPU, shared GPU, and memory-segmented. In shared GPU mode, tasks are bound to a specified physical GPU card within the node using a GPU card index list. Card-level isolation is achieved by injecting a visible GPU list during container runtime. In shared GPU mode, resource requests are not made for the entire card, but rather based on minimum CPU / memory requests to achieve multi-task reuse within the node. In memory-segmented mode, memory-segmented resource requests are generated based on memory values ​​to achieve memory-level scheduling. When a node switches to memory-segmented mode, the original node type tag is backed up first, and then the segmentation mode tag is written. When canceling memory-segmented mode or deleting a node, the original node type tag is restored. Running task verification is performed before mode switching to prevent inconsistencies between node mode changes and existing task states. "Resource group isolation" and "fine-grained GPU reuse" are integrated into the same scheduling architecture to form a hierarchical scheduling chain.

[0068] Example 4 Figure 7 This is a schematic diagram of a resource group construction device according to Embodiment 4 of the present invention. Figure 7 As shown, the device includes: an information acquisition module 71, a resource determination module 72, and a resource scheduling module 73.

[0069] Among them, the information acquisition module 71 is used to acquire business requests, determine the business type, target resource group, and node resource status of the target resource group corresponding to the business request.

[0070] The resource determination module 72 is used to determine the resource scheduling parameters corresponding to the business request based on the business type, and to determine the target node resources based on the node resource status and resource scheduling parameters; wherein, the resource scheduling parameters include the target node operating mode and resource specification parameters.

[0071] Resource scheduling module 73 is used to execute business requests based on the resources of the target node.

[0072] The technical solution of this invention involves an information acquisition module that obtains a service request, determines the service type, target resource group, and node resource status of the target resource group corresponding to the service request, a resource determination module that determines the resource scheduling parameters corresponding to the service request based on the service type, and determines the target node resources based on the node resource status and resource scheduling parameters, and a resource scheduling module that executes the service request based on the target node resources. This achieves resource isolation for different services, avoids resource contention and interference, and supports different node operating modes. It breaks the limitation of existing technologies that only support dedicated use of the entire card, improves the utilization rate of GPU resources, and reduces resource idleness caused by small-scale services.

[0073] In one embodiment, the information acquisition module 71 includes: The resource identification unit is used to receive service requests and identify the service type and target resource group corresponding to the service request. The information acquisition unit is used to extract the current remaining resources and node operating modes of nodes within the target resource group, and to use the current remaining resources and node operating modes as the node resource status.

[0074] In one embodiment, the resource determination module 72 includes: The parameter determination unit is used to extract a preset resource mapping template, match the target node operating mode and resource specification parameters corresponding to the business type in the preset resource mapping template, and use the target node operating mode and resource specification parameters as resource scheduling parameters; wherein, the target node operating mode includes exclusive mode, shared mode and video memory partitioning mode; The node determination unit is used to extract nodes whose node operating mode is the target node operating mode from the node resource status as nodes to be allocated; The resource determination unit is used to determine the nodes to be allocated that meet the resource specification parameters as target nodes, and to use the target allocation resources in the target nodes as target node resources.

[0075] In one embodiment, when the target node's operating mode is video memory partitioning mode, the resource determination unit is specifically used for: Determine the available resources of the target node, and match the resources with the same resource specification parameters as the target node resources.

[0076] In one embodiment, the resource scheduling module 73 includes: The resource scheduling unit is used to create a corresponding container runtime environment based on the target node resources and deploy the business tasks corresponding to the business requests to run in the container.

[0077] In one embodiment, the resource scheduling device further includes: Select nodes in the cluster that match the target resource specifications, and build resource groups based on the nodes; Determine the node operating mode for each node; the node operating mode includes exclusive mode, shared mode, and video memory partitioning mode. Node type labels are created for nodes according to their operating modes, and resource group labels are created for each node.

[0078] In one embodiment, the resource scheduling device further includes: If there is no node in the target resource group that operates in the target node's operating mode, extract the idle nodes in the target resource group as candidate nodes; Identify candidate nodes that support the switching of operating modes as target switching nodes, and switch the operating mode of the target switching nodes to the target node's operating mode.

[0079] The resource group construction apparatus provided in this embodiment of the invention can execute the resource group construction method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0080] Example 5 Figure 8 This is a schematic diagram of the structure of an electronic device implementing the resource group construction method of embodiments of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0081] like Figure 8 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0082] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0083] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as resource group construction methods.

[0084] In some embodiments, the resource group construction method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the resource group construction method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to execute the resource group construction method by any other suitable means (e.g., by means of firmware).

[0085] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0086] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0087] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0088] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0089] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0090] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0091] In one embodiment, the present invention further includes a computer program product, which includes a computer program that, when executed by a processor, implements the resource group construction method of any embodiment of the present invention.

[0092] In implementing the computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0093] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0094] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A resource scheduling method, characterized in that, include: Obtain the business request, determine the business type, target resource group, and node resource status of the target resource group corresponding to the business request; Based on the business type, the resource scheduling parameters corresponding to the business request are determined, and the target node resources are determined according to the node resource status and the resource scheduling parameters; wherein, the resource scheduling parameters include the target node operating mode and resource specification parameters; The service request is executed based on the resources of the target node.

2. The method according to claim 1, characterized in that, The process of obtaining a service request and determining the service type, target resource group, and node resource status of the target resource group corresponding to the service request includes: Receive a service request and identify the service type and target resource group corresponding to the service request; Extract the current remaining resources and node operating mode of the nodes within the target resource group, and use the current remaining resources and node operating mode as the node resource status.

3. The method according to claim 1, characterized in that, The step of determining the resource scheduling parameters corresponding to the service request based on the service type, and determining the target node resource based on the node resource status and the resource scheduling parameters, includes: Extract a preset resource mapping template, match the target node operating mode and resource specification parameters corresponding to the business type in the preset resource mapping template, and use the target node operating mode and the resource specification parameters as resource scheduling parameters; wherein, the target node operating mode includes exclusive mode, shared mode and video memory partitioning mode; Extract nodes whose running mode is the target node running mode from the node resource status and designate them as nodes to be allocated. Identify the nodes to be allocated that meet the resource specification parameters as target nodes, and use the target allocation resources in the target nodes as target node resources.

4. The method according to claim 3, characterized in that, When the target node is running in video memory partitioning mode, the step of using the target allocated resources in the target node as the target node resources includes: Determine the available resources of the target node, and match resources with the same resource specification parameters from the available resources as the target node resources.

5. The method according to claim 1, characterized in that, The step of executing the service request based on the target node resources includes: A corresponding container runtime environment is created based on the target node resources, and the business task corresponding to the business request is deployed to the container for execution.

6. The method according to claim 1, characterized in that, Before obtaining the service request and determining the service type, target resource group, and node resource status of the target resource group corresponding to the service request, the following steps are also included: Select nodes in the cluster that match the target resource specifications, and build resource groups based on the nodes; Determine the node operating mode for each node; wherein, the node operating mode includes exclusive mode, shared mode, and video memory partitioning mode; Node type tags are constructed for nodes according to the node operation mode, and resource group tags are created for each node.

7. The method according to claim 1, characterized in that, The resource scheduling method further includes: If there is no node in the target resource group that operates in the target node's operating mode, then extract any idle nodes in the target resource group as candidate nodes. A candidate node that supports the switching of operating modes is identified as the target switching node, and the operating mode of the target switching node is switched to the target node's operating mode.

8. A resource scheduling device, characterized in that, include: The information acquisition module is used to acquire business requests, determine the business type, target resource group, and node resource status of the target resource group corresponding to the business request; wherein, the resource scheduling parameters include the target node operating mode and resource specification parameters; The resource determination module is used to determine the resource scheduling parameters corresponding to the service request based on the service type, and to determine the target node resource based on the node resource status and the resource scheduling parameters. The resource scheduling module is used to execute the business request based on the resources of the target node.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the resource scheduling method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the resource scheduling method of any one of claims 1-7.

Images