Calculation power resource scheduling method and device, electronic equipment, medium and program product

Abstract

The invention provides a computing power resource scheduling method and device, electronic equipment, a medium and a program product, belongs to the technical field of computing power networks, and aims to improve the scheduling efficiency of complex heterogeneous computing power resources by uniformly managing nodes of various computing power resources through equipment plug-ins. And the resource scheduling request is generated according to the priority and the weight of the resource scheduling task, so that intelligent scheduling of complex heterogeneous computing power resources is realized. In addition, for the upgrading computing power resource, virtualization of the upgrading computing power resource is realized by modifying the resource scheduling request and adding the identifier of the upgrading computing power resource, and the efficiency of calling the upgrading computing power resource is improved. And for the NVIDIA computing power resource, the NVIDIA computing power resource is called according to the dynamic link library, so that the dynamic virtualization efficiency of the NVIDIA computing power resource is improved.

Description

Technical Field

[0001] This application relates to the field of computing network technology, and in particular to a method, apparatus, electronic device, medium and program product for scheduling computing resources. Background Technology

[0002] Existing computing resource scheduling includes computing resource virtualization and virtual computing resource scheduling. Computing resource virtualization includes static virtualization and dynamic virtualization.

[0003] Static virtualization resources have a fixed amount of GPU memory. If a workload exceeds the amount of GPU memory a single instance can provide, the computing resources need to be re-planned and allocated. This re-allocation will cause resources to become unusable, requiring additional human resources costs. Furthermore, configuration and maintenance are relatively complex, demanding high technical skills and extensive experience from administrators. Moreover, the management difficulty increases significantly as the number of virtual resources increases.

[0004] Dynamic virtualization introduces an additional software layer that handles tasks such as virtual resource access requests and state management, increasing processing latency and resource consumption. This overhead is particularly noticeable in high-load or complex operating environments, potentially leading to a decrease in computing performance compared to running directly on physical hardware. Furthermore, some architectures lack the computing power to support dynamic virtualization.

[0005] Virtual computing resource scheduling includes dynamic scheduling and static scheduling. Static scheduling is a method where the execution order and resource allocation are determined before a job or task is submitted. While simple and easy to implement, it cannot dynamically adjust resource allocation based on the actual needs of the application and system load. Dynamic scheduling strategies are typically simpler, unable to accurately respond to changes in application computing resource requirements or detect system load trends, thus failing to achieve fair resource allocation and scheduling, resulting in resource waste.

[0006] In cloud environments with heterogeneous computing resources, there are typically multiple types of computing resources provided by different vendors, each with its own hardware architecture and performance characteristics. This necessitates the development of specific scheduling strategies for different computing resources in practical applications, increasing the complexity of computing resource scheduling.

[0007] In summary, the existing scheduling of computing resources cannot efficiently utilize complex heterogeneous computing resources. Summary of the Invention

[0008] This application provides a method for scheduling computing resources to address the shortcomings of existing technologies in scheduling computing resources, which cannot efficiently call up complex heterogeneous computing resources, thereby improving the scheduling efficiency of complex heterogeneous computing resources.

[0009] In a first aspect, the application provides a method for scheduling computing resources, comprising: obtaining nodes of multiple computing resources of a container orchestration platform k8s, the nodes comprising a container runtime and a device plugin, and the computing resources comprising at least basic computing resources, upgraded computing resources of the basic computing resources, and Nvidia computing resources; determining an execution task based on a priority and a weight of a resource scheduling task, generating a resource scheduling request based on the execution task; when the requested resource of the resource scheduling request is the upgraded computing resources, manipulating the device plugin of the upgraded computing resources, modifying the requested resource to the basic computing resources, adding an identifier of the upgraded computing resources in the resource scheduling request, obtaining a modified resource scheduling request, and calling the container runtime of the node of the upgraded computing resources to call the upgraded computing resources based on the modified resource scheduling request; and when the requested resource of the resource scheduling request is the Nvidia computing resources, obtaining the device plugin of the node of the Nvidia computing resources based on the resource scheduling request, and calling the Nvidia computing resources based on a dynamic link library of the container runtime of the node of the Nvidia computing resources.

[0010] In one embodiment, the priority comprises a low priority and a high priority, and determining the execution task based on the priority and the weight of the resource scheduling task comprises: when the priority of the new resource scheduling task is the high priority and no node matching the new resource scheduling task is found, releasing a node of a resource scheduling task being executed with the low priority to obtain a released task; adding the new resource scheduling task and the released task to a waiting queue to obtain waiting tasks; and determining the weight of the waiting tasks based on a task running time and a task scheduling number in one scheduling period, and sequentially scheduling the waiting tasks based on the size of the weight to obtain the execution task, wherein the task scheduling number represents a number of scheduling periods passed by the waiting tasks.

[0011] In one embodiment, after calling the container runtime of the node of the upgraded computing resources to call the upgraded computing resources based on the modified resource scheduling request, the method further comprises: when a running computing resource of a container of the execution task is less than a set percentage of a computing resource occupied by the execution task, marking the container of the execution task as a low computing container; and causing multiple low computing containers to collectively occupy one computing resource.

[0012] In an embodiment, after the dynamic link library of the container runtime of the node based on the Nvidia computing resource calls the Nvidia computing resource, the method further comprises: when the required computing power of a first target execution task of the Nvidia computing resource is greater than a first set computing power, and the priorities of other execution tasks of the Nvidia computing resource are all lower than the priority of the first target execution task, taking the total computing power of the Nvidia computing resource as the required computing power of the first target execution task; when the required computing power of a second target execution task of the Nvidia computing resource is less than a second set computing power, taking the computing power of one Nvidia computing resource as the required computing power of a plurality of second target execution tasks, and the sum of the required computing power of the plurality of second target execution tasks is greater than the computing power of one Nvidia computing resource.

[0013] In an embodiment, the dynamic link library of the container runtime of the node based on the Nvidia computing resource calls the Nvidia computing resource, comprising: based on the interaction result of the dynamic link library and the container runtime, obtaining the video memory space of the container of the Nvidia computing resource; verifying the video memory space of the video memory request of the dynamic link library to the Nvidia computing resource to limit the video memory space of the video memory request within the video memory space of the container, the video memory request being sent by the CUDA Runtime of the unified computing device architecture runtime of the Nvidia computing resource to the CUDADriver of the unified computing device architecture of the Nvidia computing resource; calling the CUDAAPI of the unified computing device architecture based on the verified video memory request to call the Nvidia computing resource.

[0014] In an embodiment, the node further comprises a label, an annotation and a monitoring unit: the label is used to identify the node; the annotation is used to record the identifier of the computing resource under the node, the model of the computing resource under the node and the video memory of the computing resource under the node; the monitoring unit is used to report the usage of the computing resource under the node to the monitoring system of the k8s.

[0015] In a second aspect, the application also provides a computing resource scheduling apparatus, comprising: an acquisition module configured to acquire nodes of multiple computing resources of a container orchestration platform k8s, wherein the nodes comprise a container runtime and a device plugin, and the computing resources at least include basic computing resources, upgraded computing resources of the basic computing resources, and NVIDIA computing resources; a determination module configured to determine an execution task based on a priority and a weight of a resource scheduling task, and generate a resource scheduling request based on the execution task; a first scheduling module configured to, when a requested resource of the resource scheduling request is the upgraded computing resources, manipulate the device plugin of the upgraded computing resources, modify the requested resource into the basic computing resources, add an identifier of the upgraded computing resources in the resource scheduling request, obtain a modified resource scheduling request, and invoke the container runtime of the node of the upgraded computing resources to invoke the upgraded computing resources based on the modified resource scheduling request; and a second scheduling module configured to, when the requested resource of the resource scheduling request is the NVIDIA computing resources, acquire the device plugin of the node of the NVIDIA computing resources based on the resource scheduling request, and invoke the NVIDIA computing resources based on a dynamic link library of the container runtime of the node of the NVIDIA computing resources.

[0016] In a third aspect, the application also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements any of the above computing resource scheduling methods when executing the computer program.

[0017] In a fourth aspect, the application also provides a non-transitory computer readable storage medium having a computer program stored thereon, and the computer program is executable on a processor to implement any of the above computing resource scheduling methods.

[0018] In a fifth aspect, the application also provides a computer program product comprising a computer program, and the computer program is executable on a processor to implement any of the above computing resource scheduling methods.

[0019] The computing resource scheduling method, apparatus, electronic device, medium, and program product provided by the application can uniformly manage the nodes of multiple computing resources through the device plugin, thereby improving the scheduling efficiency of complex heterogeneous computing resources. The resource scheduling request is generated based on the priority and weight of the resource scheduling task, thereby realizing intelligent scheduling of complex heterogeneous computing resources. For the upgraded computing resources, the resource scheduling request is modified and the identifier of the upgraded computing resources is added, thereby realizing virtualization of the upgraded computing resources and improving the efficiency of invoking the upgraded computing resources. For the NVIDIA computing resources, the NVIDIA computing resources are invoked based on the dynamic link library, thereby improving the dynamic virtualization efficiency of the NVIDIA computing resources. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the accompanying drawings in the following description are some embodiments of the application, and for those skilled in the art, other drawings can also be obtained based on these drawings without creative labor.

[0021] Figure 1 is one of the flowcharts of the scheduling method of the computing resource provided by the application.

[0022] Figure 2 is another flowchart of the scheduling method of the computing resource provided by the application.

[0023] Figure 3 is one of the structural diagrams of the node of the computing resource provided by the application.

[0024] Figure 4 is another structural diagram of the node of the computing resource provided by the application.

[0025] Figure 5 is the flowchart of the scheduling method of Atlas 300I Duo provided by the application.

[0026] Figure 6 is the flowchart of the scheduling method of Nvidia computing resource provided by the application.

[0027] Figure 7 is the third flowchart of the scheduling method of the computing resource provided by the application.

[0028] Figure 8 is the flowchart of the recombination of the low computing container provided by the application.

[0029] Figure 9 is the third flowchart of the scheduling method of the computing resource provided by the application.

[0030] Figure 10 is the structural diagram of the scheduling device of the computing resource provided by the application.

[0031] Figure 11 is the structural diagram of the electronic device provided by the application. DETAILED DESCRIPTION

[0032] In order to make the purposes, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in combination with the drawings in the present application. Obviously, the described embodiments are some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0033] The technical solutions of the present application will be described below in combination with the drawings in the present application. Figures 1-11 The scheduling method, device and electronic equipment of computing resources are described in the present application.

[0034] Figure 1 is one of the flowcharts of the scheduling method of computing resources provided by the present application, as shown in Figure 1 The scheduling method of computing resources includes S100 to S400, and each step is as follows.

[0035] S100: Obtain nodes of multiple computing resources of a container orchestration platform k8s.

[0036] The nodes include a container runtime and a device plugin, and the computing resources at least include basic computing resources, upgraded computing resources of the basic computing resources and NVIDIA computing resources.

[0037] Based on the above embodiments, the nodes further include labels, annotations and a monitoring unit: the labels are used to identify the nodes; the annotations are used to record identifiers of the computing resources under the nodes, models of the computing resources under the nodes and video memories of the computing resources under the nodes; and the monitoring unit is used to report usage of the computing resources under the nodes to a monitoring system of the k8s.

[0038] The computing resources of the present application include diversified heterogeneous computing resources. The computing resources include NVIDIA computing resources, such as NVIDIA graphics processing units (GPUs), Ascend neural processing units (NPUs), such as Atlas 300I Duo and Atlas 300I Pro, Cambrian machine learning units (MLUs), Haiguang data center units (DCUs) and Tianchuang intelligent chip GPUs.

[0039] The application innovatively provides a unified device plugin or registration interface to support a heterogeneous computing power resource mixed deployment scenario, and provides a unified computing power resource registration template (a unified interface template) for a virtualized heterogeneous computing power resource scenario. The device plugin of the node of the virtualized computing power resource can be dynamically added to a cluster for unified scheduling management after implementing registration logic according to the computing power resource registration template.

[0040] As shown in Figure 3 , the node of the computing power resource installs a corresponding device driver and a container runtime (Container Runtime), and injects a computing power resource virtualization technology through the Container Runtime. The node is added to a k8s cluster, and a label (Node Label) matched with the corresponding computing power resource is created. A device plugin (DevicePlugin) of the node of the computing power resource is registered, and the name of the registered computing power resource is added to the configuration of a scheduler component of the k8s. A monitoring (monitor) unit of the node is set. The monitoring unit is used to report the use of the computing power resource under the node to a monitoring system (Prometheus) of the k8s. The device plugin writes a Universally Unique Identifier (UUID) of the computing power resource, a model of the computing power resource, and detailed information such as a display memory into an annotation (Node Annotation) of the node. As shown in Figure 4 , a node of a gpu is constructed, and the node of the gpu includes a device plugin of a virtual gpu (vgpu) and a monitoring unit of the vgpu. A node of an npu is constructed, and the node of the npu includes a device plugin of a virtual npu (vnpu) and a monitoring unit of the vnpu. Further, according to the unified registration interface, a plugin of other types of computing power resources is extended, and then a node of a dpu, a node of an mlu, and nodes of other types of computing power resources are constructed.

[0041] A scheduler (a scheduler node) of the k8s updates a list of currently available computing power resources (Devices) through all annotations of the nodes of the computing power resources before a scheduling period.

[0042] As shown in Figure 4 , the device plugin, the monitoring unit, and the scheduler component are highly integrated through a k8s package manager (helm package). Different nodes of the computing power resources in the k8s are distinguished through labels, and the helm package installation command is used to realize cloud deployment of a quick heterogeneous computing power resource scenario.

[0043] The application integrates the computing power resource monitoring, the virtualization, and the scheduler into a helm package, and a user only needs to use a simple command to complete the quick cloud deployment of the application.

[0044] As shown in Figure 2 k8s uses a unified web page hook (Webhook) to listen to resource scheduling tasks (Pod requests).

[0045] S200: Determine the execution task based on the priority and weight of the resource scheduling task, and generate a resource scheduling request based on the execution task.

[0046] When receiving a resource scheduling task, the intelligent scheduling strategy registered by the scheduling framework is used to schedule and allocate computing resources. The intelligent scheduling strategy includes adding the resource scheduling task to the waiting queue, selecting the execution task in the waiting queue according to the priority and weight of the resource scheduling task, and generating a resource scheduling request according to the execution task. The resource scheduling request is used to request computing resources.

[0047] The application uses a pluggable scheduling framework (Scheduling Framework) as an extension of the scheduler, and registers the intelligent scheduling strategy in the form of a plug-in to dynamically schedule computing resources, realizing flexible expansion of the intelligent scheduling strategy.

[0048] At the same time, by using the monitoring service (Service Monitor) provided by the Prometheus operator, the related data indicators of the computing resources are reported to the Prometheus, including the temperature, power and memory usage of the computing resources, the memory usage of the virtual computing resources, and the number of tasks running on the computing resources. These related data indicators are fed back to the intelligent scheduling strategy, and the intelligent scheduling strategy continuously optimizes the scheduling logic according to these related data indicators.

[0049] S300: When the requested resource of the resource scheduling request is an upgraded computing resource, the device plug-in of the upgraded computing resource is controlled, the requested resource is modified to a basic computing resource, the identifier of the upgraded computing resource is added in the resource scheduling request, the modified resource scheduling request is obtained, and the container runtime of the node of the upgraded computing resource is called to call the upgraded computing resource based on the modified resource scheduling request.

[0050] The basic computing resource is a computing resource that can be dynamically virtualized in k8s, for example, Atlas 300I Pro. The upgraded computing resource is a computing resource that is insufficient in the dynamic virtualization process of k8s, for example, Atlas 300I Duo. The upgraded computing resource is improved based on the basic computing resource. The applicant has conducted in-depth and detailed research on the underlying architecture of the Atlas 300I Duo processor. By comparing the hardware design, instruction set, memory management, and device driver of Atlas 300I Duo and Atlas 300I Pro, the similarities in the underlying implementation of the two are found. According to the similarity, the dynamic virtualization process of Atlas 300I Duo is improved.

[0051] A split template of the same type as Atlas 300I Pro is constructed in the node of the computing resource, which contains the identification of Atlas 300I Duo and actually points to Atlas 300I Duo, thereby realizing the dynamic virtualization of Atlas 300I Duo.

[0052] As shown in Figure 5 When the device plugin of the virtual NPU (vnpu) corresponding to the node of Atlas 300I Duo detects that the requested resource of the resource scheduling request of the upper application is an upgraded computing resource (Atlas 300I Duo), it will first intercept the resource scheduling request, modify the requested resource (for example, the model) of the resource scheduling request to a basic computing resource (Atlas 300I Pro). At the same time, the identification of the upgraded computing resource is added in the resource scheduling request, for example, vdie id: Atlas 300I Duo is added in the resource scheduling request, indicating that the modified resource scheduling request is pointing to Atlas 300I Duo. Based on the modified resource scheduling request, the container runtime of the node of the upgraded computing resource accesses the upgraded computing resource (Atlas 300I Duo).

[0053] The application realizes the function of dynamic virtualization of Atlas 300I Duo, so that several different types of graphics cards of Atlas 910 series, 300I Pro and 300I Duo can be uniformly managed and scheduled by dynamic virtualization, greatly improving the management efficiency of k8s.

[0054] S400: When the requested resource of the resource scheduling request is a NVIDIA computing resource, the device plugin of the node of the NVIDIA computing resource is obtained based on the resource scheduling request, and the NVIDIA computing resource is called based on the dynamic link library of the container runtime of the node of the NVIDIA computing resource.

[0055] The dynamic link library of the container runtime of the node based on the NVIDIA computing resource calls the NVIDIA computing resource, including: based on the interaction result of the dynamic link library and the container runtime, obtaining the video memory space of the container of the NVIDIA computing resource; based on the video memory space of the video memory request of the dynamic link library to the NVIDIA computing resource, verifying the video memory space to limit the video memory space of the video memory request within the video memory space of the container, the video memory request being sent by the CUDA Runtime of the unified computing device architecture runtime of the NVIDIA computing resource to the CUDA Driver of the unified computing device architecture of the NVIDIA computing resource; based on the verified video memory request, calling the CUDA API of the unified computing device architecture, and calling the NVIDIA computing resource.

[0056] The existing GPU virtualization generates a virtual GPU (vGPU) through a virtualization management program, and then mounts the vGPU to a container. The CUDA API of the unified computing device architecture is called, and then the vGPU is operated to call the computing resource.

[0057] As shown in Figure 6 For the NVIDIA computing resource (NVIDIA GPU), the GPU physical hardware corresponding to the NVIDIA computing resource and the dynamic link library (libvgpu.so) are directly mounted to the container, the call of the CUDA API by the user code directly accesses the physical GPU through the libvgpu.so, and the limitation and isolation of the computing resource are controlled by the libvgpu.so in real time.

[0058] According to the interaction result of the dynamic link library and the container runtime (Container Runtime), the video memory space of the container of the NVIDIA computing resource is obtained. When the CUDA Runtime sends a video memory request to the CUDA Driver, the video memory request is verified to limit the video memory space of the video memory request within the video memory space of the container. Based on the verified video memory request, the CUDA API is called, and then the NVIDIA computing resource is called.

[0059] According to the application, the video memory space of the video memory request of the dynamic link library (libvgpu.so) to the NVIDIA computing resource is verified to limit the video memory space of the video memory request within the video memory space of the container, the step of generating a vGPU is omitted, and the dynamic virtualization efficiency of the NVIDIA computing resource is improved. Through the direct mounting manner of the NVIDIA computing resource, a more flexible dynamic expansion method is provided.

[0060] The dynamic link library (libvgpu.so) of the application can intercept the memory request between the CUDA Runtime and the CUDA Driver, thereby hijacking the CUDA API, realizing the direct control and access of the hardware of the Nvidia computing power resource, and can be used without system root (root) permission, and the Nvidia computing power resource (Nvidia GPU) can be dynamically virtualized flexibly, and the access call to the virtualized Nvidia GPU has almost no performance loss.

[0061] For virtualization of various computing power resources such as Cambrian MLU, Haiguang DCU, and Tianriu GPU, because the manufacturers do not disclose the source code of the underlying driver library, the Container Runtime provided by the manufacturer directly integrated through the unified Device Plugin interface realizes the unified management of the scheduler.

[0062] The scheduling method of the computing power resource provided by the embodiment of the application unifies the management of the nodes of various computing power resources through the device plug-in, improves the scheduling efficiency of complex heterogeneous computing power resources, generates a resource scheduling request according to the priority and weight of the resource scheduling task, realizes intelligent scheduling of complex heterogeneous computing power resources, modifies the resource scheduling request and adds the identifier of the upgraded computing power resource for the upgraded computing power resource, realizes the virtualization of the upgraded computing power resource, and is beneficial to improving the efficiency of the call of the upgraded computing power resource. For the Nvidia computing power resource, the Nvidia computing power resource is called according to the dynamic link library, and the dynamic virtualization efficiency of the Nvidia computing power resource is improved.

[0063] Based on the above embodiment, the priority includes a low priority and a high priority, the execution task is determined based on the priority and the weight of the resource scheduling task, and includes S210 to S230, and each step is specifically as follows.

[0064] S210: When the priority of the new resource scheduling task is a high priority, and a node matched with the new resource scheduling task is not found, a node of a low-priority executing resource scheduling task is released to obtain a released task.

[0065] S220: The new resource scheduling task and the released task are added to a waiting queue to obtain a waiting task.

[0066] S230: In a scheduling period, the weight of the waiting task is determined based on the task running time and the task scheduling times, the waiting tasks are sequentially scheduled based on the size of the weight to obtain an execution task, and the task scheduling times represent the number of scheduling periods passed by the waiting task.

[0067] The application introduces a priority management mechanism to ensure that important tasks (high-priority resource scheduling tasks) can obtain sufficient computing power resources. Administrators can set different priorities for resource scheduling tasks according to the urgency and importance of the tasks, and the scheduler will give priority to high-priority resource scheduling tasks when allocating computing power resources. If the application lacks computing power resources, the scheduler will first evaluate the usage of computing power resources on all nodes and try to find a node that meets the requirements of the new resource scheduling task. If no node meets the requirements, the scheduler will enter the preemption mode and start considering expelling some low-priority resource scheduling tasks. The expulsion decision is based on multiple factors, such as the priority of the resource scheduling task, the amount of computing power resources used, the expulsion cost, etc. Ensuring that high-priority resource scheduling tasks can obtain sufficient resources improves the stability and reliability of the system.

[0068] As shown in Figure 7 , the webhook listens for new resource scheduling tasks and determines the requested resources of the new resource scheduling task. It is determined whether the requested resource exists in the computing power resource list. If the requested resource exists in the computing power resource list, the intelligent scheduling strategy is executed. If the requested resource does not exist in the computing power resource list, the scheduling is completed directly (the scheduling ends).

[0069] It is determined whether the new resource scheduling task needs to wait, and if it needs to wait, priority scheduling is performed. When the priority of the new resource scheduling task is high and no node matching the new resource scheduling task is found, the node executing the low-priority resource scheduling task is released, and the released task is obtained. The new resource scheduling task and the released task are added to the waiting queue, and the waiting task is obtained.

[0070] For the waiting task in the waiting queue, in a scheduling period, the weight of each waiting task is determined according to the task running time and the task scheduling times. The weight of the waiting task = task scheduling times / task running time. The shorter the task running time, the higher the weight of the waiting task. The task scheduling times of the waiting task are recorded, and if the waiting task has not completed scheduling after multiple scheduling periods, the task scheduling times are increased, and the weight of the waiting task is also increased accordingly. The weights of the waiting tasks are sorted, and one or more waiting tasks with high weights are selected as the execution task of the scheduling period according to the sorting result.

[0071] Further, as shown in Figure 9As shown, when the waiting task is taken out from the waiting queue to obtain the execution task, the scheduler classifies and schedules the execution task according to the description of the corresponding service capability and the application resource (required computing resource) of the execution task according to the high and low of the computing performance requirement. First, the scheduler calculates the load of all computing resource nodes and sorts them. For the execution task requiring high computing performance, the scheduler preferentially selects a cluster of nodes with sufficient computing resources and low load, and individually calls the computing resources in the cluster to ensure that the execution task can be completed in the shortest time. For the execution task with low computing performance, the scheduler adopts device multiplexing to allocate multiple execution tasks to the same computing resource for execution to improve the utilization rate of the computing resource.

[0072] The present application sets priority to ensure that high-priority resource scheduling tasks can obtain sufficient computing resources, thereby improving the reliability and stability of computing resource scheduling. According to the weight, the waiting task is scheduled, which improves the execution efficiency of the waiting task and ensures the fairness of the waiting task scheduling.

[0073] Based on the above embodiment, based on the modified resource scheduling request, the container runtime of the node of the upgraded computing resource is called to upgrade the computing resource, and S500 to S600 are further included after the computing resource is upgraded, and each step is specifically as follows.

[0074] S500: When the running computing resource of the container of the execution task is less than the set percentage of the computing resource occupied by the execution task, the container of the execution task is marked as a low-computing container.

[0075] S600: A plurality of low-computing containers collectively occupy one computing resource.

[0076] During the long-time running of k8s, due to the frequent creation and destruction of container groups (Pods), resource fragmentation problems may occur in the cluster of computing resource nodes. For example, Figure 8As shown, the scheduler can periodically scan the computing power resource usage in the cluster, identify nodes or regions with more fragmentation, filter out low computing containers, and try to reduce computing power resource fragmentation (computing power resources with usage lower than the set usage) by rescheduling these low computing containers, concentrate multiple low computing containers on one computing power resource (for example, upgrade the computing power resource), improve the overall utilization of the computing power resource and the performance of the cluster, and also ensure that the cluster has enough resources to guarantee the execution of resource scheduling tasks with high resource requirements. At the same time, the rescheduling of computing power resource fragmentation uses a rolling update method. Rolling update is a strategy to update an application by replacing old version components one by one. It updates a small part of the computing power resource fragmentation at a time, and if it succeeds, it updates more computing power resource fragmentation, and finally updates all computing power resource fragmentation. The main advantage of this update method is that it realizes zero downtime, ensuring the continuity and high availability of the service.

[0077] The present application realizes the reorganization of low computing containers by concentrating multiple low computing containers on one computing power resource, reduces computing power resource fragmentation, and is beneficial to improve the use efficiency of computing power resources.

[0078] Based on the above embodiment, the dynamic link library of the container runtime of the node based on the NVIDIA computing power resource calls the NVIDIA computing power resource, and further includes S700-S800, and each step is specifically as follows.

[0079] S700: When the demand computing power of the first target execution task of the NVIDIA computing power resource is greater than the first set computing power, and the priorities of other execution tasks of the NVIDIA computing power resource are all lower than the priority of the first target execution task, the total computing power of the NVIDIA computing power resource is used as the demand computing power of the first target execution task.

[0080] S800: When the demand computing power of the second target execution task of the NVIDIA computing power resource is less than the second set computing power, the computing power of one NVIDIA computing power resource is used as the demand computing power of multiple second target execution tasks, and the sum of the demand computing power of multiple second target execution tasks is greater than the computing power of one NVIDIA computing power resource.

[0081] When the requested resource of the execution task is the NVIDIA computing power resource (for example, NVIDIA GPU), the scheduling strategy optimizes the allocation of computing power resources for execution tasks with high computing performance (first target execution task) and execution tasks with low computing performance (second target execution task), in order to further improve the utilization of computing power resources.

[0082] When the demand computing power of the first target execution task is greater than the first set computing power, the first target execution task is set as a high computing performance requirement execution task. When there is no other same or higher priority execution task on the (used) Nvidia computing power resource corresponding to the high computing performance requirement execution task, the restriction (80% of the computing power resource) on the Nvidia computing power resource is released, and the entire computing power (100%) of the Nvidia computing power resource is used as the demand computing power of the high computing performance requirement execution task, so as to realize computing power expansion.

[0083] When the demand computing power of the second target execution task is less than the second set computing power, the second target execution task is set as a low computing performance execution task. Multiple low computing performance execution tasks are allocated to one Nvidia computing power resource, so that the computing power allocation amount of the Nvidia computing power resource exceeds 100% (the sum of the demand computing power of multiple execution tasks is greater than the computing power of one Nvidia computing power resource), so as to realize computing power oversubscription.

[0084] The application uses the entire computing power of the Nvidia computing power resource as the demand computing power for the first target execution task, so as to realize computing power expansion and improve the use efficiency of the Nvidia computing power resource. For the second target execution task, the computing power of one Nvidia computing power resource is used as the demand computing power of multiple second target execution tasks, so as to realize computing power oversubscription and improve the use efficiency of the Nvidia computing power resource.

[0085] The power resource scheduling device provided in the application is described below, and the power resource scheduling device described below can be correspondingly referred to the power resource scheduling method described above. As shown in Figure 10 The power resource scheduling device includes an acquisition module 1001 configured to acquire nodes of multiple power resources of a container orchestration platform k8s, the nodes including a container runtime and a device plug-in, and the power resources including at least a basic power resource, an upgraded power resource of the basic power resource, and a Nvidia computing power resource.

[0086] A determination module 1002 is configured to determine an execution task based on the priority and weight of a resource scheduling task, and generate a resource scheduling request based on the execution task.

[0087] A first scheduling module 1003 is configured to, when the requested resource of the resource scheduling request is the upgraded power resource, manipulate the device plug-in of the upgraded power resource, modify the requested resource into the basic power resource, add an identifier of the upgraded power resource in the resource scheduling request, obtain a modified resource scheduling request, and call the container runtime of the node of the upgraded power resource to call the upgraded power resource based on the modified resource scheduling request.

[0088] The second scheduling module 1004 is configured to, when the requested resource of the resource scheduling request is the NVIDIA computing resource, obtain a device plugin of a node of the NVIDIA computing resource based on the resource scheduling request, and call the NVIDIA computing resource based on a dynamic link library of a container runtime of the node of the NVIDIA computing resource.

[0089] The scheduling device for computing resources provided by the embodiments of the present application can uniformly manage the nodes of various computing resources through the device plugin, thereby improving the scheduling efficiency of complex heterogeneous computing resources. The resource scheduling request is generated according to the priority and weight of the resource scheduling task, thereby realizing intelligent scheduling of complex heterogeneous computing resources. For the upgraded computing resource, the virtualization of the upgraded computing resource is realized by modifying the resource scheduling request and adding the identifier of the upgraded computing resource, which is beneficial to improving the efficiency of calling the upgraded computing resource. For the NVIDIA computing resource, the NVIDIA computing resource is called according to the dynamic link library, thereby improving the dynamic virtualization efficiency of the NVIDIA computing resource.

[0090] In one embodiment, the priority includes a low priority and a high priority, and the determining module 1002 is configured to, when the priority of the new resource scheduling task is the high priority and no node matched with the new resource scheduling task is found, release the node of the executing resource scheduling task with the low priority to obtain a released task, add the new resource scheduling task and the released task to a waiting queue to obtain a waiting task, and determine the weight of the waiting task based on the task running time and the task scheduling times in one scheduling period, and sequentially schedule the waiting task based on the size of the weight to obtain an executing task, wherein the task scheduling times represent the number of scheduling periods passed by the waiting task.

[0091] In one embodiment, the first scheduling module 1003 is further configured to, when the running computing resource of the container of the executing task is less than the set percentage of the computing resource occupied by the executing task, mark the container of the executing task as a low-computing container, and cause the plurality of low-computing containers to collectively occupy one computing resource.

[0092] In one embodiment, the second scheduling module 1004 is further configured to, when the demand computing resource of a first target executing task of the NVIDIA computing resource is greater than a first set computing resource, and the priorities of other executing tasks of the NVIDIA computing resource are all lower than the priority of the first target executing task, take the total computing resource of the NVIDIA computing resource as the demand computing resource of the first target executing task, and when the demand computing resource of a second target executing task of the NVIDIA computing resource is less than a second set computing resource, take the computing resource of one NVIDIA computing resource as the demand computing resource of a plurality of second target executing tasks, and the sum of the demand computing resources of the plurality of second target executing tasks is greater than the computing resource of one NVIDIA computing resource.

[0093] In one embodiment, the first scheduling module 1003 is configured to: based on the interaction result of the dynamic link library and the container runtime, obtain the video memory space of the container of the Nvidia computing resource; based on the video memory request of the dynamic link library to the video memory of the Nvidia computing resource, verify the video memory space to limit the video memory space of the video memory request within the video memory space of the container, the video memory request being sent by the CUDA Runtime of the unified computing device architecture runtime of the Nvidia computing resource to the CUDA Driver of the unified computing device architecture of the Nvidia computing resource; based on the verified video memory request, call the CUDA API of the unified computing device architecture to call the Nvidia computing resource.

[0094] In one embodiment, the node further includes a label, an annotation and a monitoring unit: the label is used to identify the node; the annotation is used to record the identifier of the computing resource under the node, the model of the computing resource under the node and the video memory of the computing resource under the node; the monitoring unit is used to report the usage of the computing resource under the node to the monitoring system of the k8s.

[0095] Figure 11 An example of a schematic diagram of the physical structure of an electronic device is shown in Figure 11 As shown, the electronic device can include a processor 1110, a communications interface 1120, a memory 1130 and a communications bus 1140, wherein the processor 1110, the communications interface 1120 and the memory 1130 complete mutual communication through the communications bus 1140. The processor 1110 can invoke the logical instructions in the memory 1130 to execute the scheduling method of the computing resource, which includes: obtaining the nodes of multiple computing resources of a container orchestration platform k8s, the nodes including a container runtime and a device plugin, and the computing resources including at least a basic computing resource, an upgraded computing resource of the basic computing resource and a Nvidia computing resource; determining an execution task based on the priority and weight of a resource scheduling task, generating a resource scheduling request based on the execution task; when the requested resource of the resource scheduling request is the upgraded computing resource, manipulating the device plugin of the upgraded computing resource, modifying the requested resource to the basic computing resource, adding the identifier of the upgraded computing resource in the resource scheduling request to obtain a modified resource scheduling request, and calling the container runtime of the node of the upgraded computing resource to call the upgraded computing resource based on the modified resource scheduling request; when the requested resource of the resource scheduling request is the Nvidia computing resource, obtaining the device plugin of the node of the Nvidia computing resource based on the resource scheduling request, and calling the Nvidia computing resource based on the dynamic link library of the container runtime of the node of the Nvidia computing resource.

[0096] Further, the logic instructions in the memory 1130 described above can be implemented in the form of software function units and sold or used as independent products, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0097] In another aspect, the present application also provides a computer program product, the computer program product comprising a computer program, the computer program being stored on a non-transitory computer readable storage medium, and the computer program being executable by a processor to cause a computer to perform the method of scheduling the computing resource provided by the above-mentioned methods, the method comprising: obtaining nodes of a plurality of computing resources of a container orchestration platform k8s, the nodes comprising a container runtime and a device plugin, and the computing resources comprising at least basic computing resources, upgraded computing resources of the basic computing resources, and Nvidia computing resources; determining an execution task based on a priority and a weight of a resource scheduling task, and generating a resource scheduling request based on the execution task; when the requested resource of the resource scheduling request is the upgraded computing resources, manipulating the device plugin of the upgraded computing resources, modifying the requested resource to the basic computing resources, adding an identifier of the upgraded computing resources in the resource scheduling request, obtaining a modified resource scheduling request, and calling the container runtime of the node of the upgraded computing resources to call the upgraded computing resources based on the modified resource scheduling request; when the requested resource of the resource scheduling request is the Nvidia computing resources, obtaining the device plugin of the node of the Nvidia computing resources based on the resource scheduling request, and calling the Nvidia computing resources based on a dynamic link library of the container runtime of the node of the Nvidia computing resources.

[0098] In yet another aspect, the present application also provides a non-transitory computer-readable storage medium having stored thereon a computer program, which, when executed by a processor, implements the method for scheduling computing resource provided by each of the above methods, and the method comprises: obtaining nodes of multiple computing resources of a container orchestration platform k8s, the nodes comprising a container runtime and a device plugin, and the computing resources at least including basic computing resources, upgraded computing resources of the basic computing resources, and Nvidia computing resources; determining an execution task based on a priority and a weight of a resource scheduling task, and generating a resource scheduling request based on the execution task; when the requested resource of the resource scheduling request is the upgraded computing resource, manipulating the device plugin of the upgraded computing resource, modifying the requested resource into the basic computing resource, adding an identifier of the upgraded computing resource in the resource scheduling request, obtaining a modified resource scheduling request, and calling the container runtime of the node of the upgraded computing resource to call the upgraded computing resource based on the modified resource scheduling request; and when the requested resource of the resource scheduling request is the Nvidia computing resource, obtaining the device plugin of the node of the Nvidia computing resource based on the resource scheduling request, and calling the Nvidia computing resource based on a dynamic link library of the container runtime of the node of the Nvidia computing resource.

[0099] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the present embodiment scheme according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0100] From the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be realized by means of software and necessary general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of software product, which can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, server, or network device, etc.) execute the method described in each embodiment or some parts of the embodiment.

[0101] Finally, it should be noted that the above examples are only used to illustrate the technical solutions of the present application, and are not intended to limit the same; although the present application has been described in detail with reference to the foregoing examples, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for scheduling computing resources, characterized in that, include: A node that acquires various computing resources from the container orchestration platform Kubernetes, the node including container runtime and device plugins, the computing resources including at least basic computing resources, upgraded computing resources of the basic computing resources and NVIDIA computing resources; The execution task is determined based on the priority and weight of the resource scheduling task, and a resource scheduling request is generated based on the execution task. When the requested resource in the resource scheduling request is the upgraded computing power resource, the device plugin that controls the upgraded computing power resource modifies the requested resource to the basic computing power resource, adds the identifier of the upgraded computing power resource to the resource scheduling request, and obtains the modified resource scheduling request. Based on the modified resource scheduling request, the container runtime of the node that calls the upgraded computing power resource calls the upgraded computing power resource. When the requested resource in the resource scheduling request is the NVIDIA computing power resource, the device plugin of the node that obtains the NVIDIA computing power resource based on the resource scheduling request, and the dynamic link library of the container runtime of the node that obtains the NVIDIA computing power resource call the NVIDIA computing power resource.

2. The method for scheduling computing resources according to claim 1, characterized in that, The priority includes low priority and high priority, and the determination of the execution task based on the priority and weight of the resource scheduling task includes: When a new resource scheduling task has a high priority and no matching node is found, the node of the low-priority currently executing resource scheduling task is released, and the task is released. The newly added resource scheduling task and the release task are added to the waiting queue to obtain the waiting task; Within a scheduling cycle, the weight of the waiting task is determined based on the task execution time and the number of task scheduling cycles. The waiting tasks are then scheduled sequentially based on the weight to obtain the execution task. The number of task scheduling cycles represents the number of scheduling cycles that the waiting task has gone through.

3. The method for scheduling computing resources according to claim 1, characterized in that, After the container runtime of the node that calls the upgrade computing resources based on the modified resource scheduling request calls the upgrade computing resources, it also includes: When the computing power resources of the container executing the task are less than a set percentage of the computing power resources occupied by the task, the container executing the task is marked as a low computing container. Multiple low-computing containers are collectively used to occupy one computing resource.

4. The method for scheduling computing resources according to claim 1, characterized in that, After the dynamic link library of the container runtime of the node based on NVIDIA computing resources calls the NVIDIA computing resources, it also includes: When the computing power required for the first target task of the NVIDIA computing power resources is greater than the first set computing power, and the priority of other tasks of the NVIDIA computing power resources is lower than the priority of the first target task, the total computing power of the NVIDIA computing power resources shall be used as the computing power required for the first target task. When the computing power required for the second target task of the NVIDIA computing power resource is less than the second set computing power, the computing power of one NVIDIA computing power resource is used as the computing power required for multiple second target tasks, and the sum of the computing power required for multiple second target tasks is greater than the computing power of one NVIDIA computing power resource.

5. The method for scheduling computing resources according to claim 1, characterized in that, The dynamic link library of the container runtime of the node based on NVIDIA computing resources calls the NVIDIA computing resources, including: Based on the interaction results between the dynamic link library and the container runtime, the video memory space of the container of the NVIDIA computing power resources is obtained. The memory space of the video memory request of the NVIDIA computing power resource is verified based on the dynamic link library to limit the video memory space of the video memory request to the memory space of the container. The video memory request is sent by the CUDA Runtime of the NVIDIA computing power resource to the CUDA Driver of the NVIDIA computing power resource. Based on the verified memory request, the CUDA API (Unified Computing Device Architecture) is invoked to access the NVIDIA computing resources.

6. The method for scheduling computing resources according to claim 1, characterized in that, The node also includes tags, annotations, and monitoring units: The label is used to identify the node; The annotation is used to record the identifier of the computing resources under the node, the model of the computing resources under the node, and the video memory of the computing resources under the node; The monitoring unit is used to report the usage of computing resources under the node to the Kubernetes monitoring system.

7. A computing resource scheduling device, characterized in that, include: The acquisition module is used to acquire nodes of various computing resources of the container orchestration platform Kubernetes. The nodes include container runtime and device plugins. The computing resources include at least basic computing resources, upgraded computing resources of the basic computing resources, and NVIDIA computing resources. The determination module is used to determine the execution task based on the priority and weight of the resource scheduling task, and generate a resource scheduling request based on the execution task; The first scheduling module is used to manipulate the device plug-in of the upgraded computing power resource when the requested resource in the resource scheduling request is the upgraded computing power resource, modify the requested resource to the basic computing power resource, add the identifier of the upgraded computing power resource to the resource scheduling request, obtain the modified resource scheduling request, and call the container runtime of the node that calls the upgraded computing power resource to call the upgraded computing power resource based on the modified resource scheduling request. The second scheduling module is used to, when the requested resource in the resource scheduling request is the NVIDIA computing power resource, obtain the device plugin of the node with the NVIDIA computing power resource based on the resource scheduling request, and call the NVIDIA computing power resource based on the dynamic link library of the container runtime of the node with the NVIDIA computing power resource.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the computing resource scheduling method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the computing resource scheduling method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the computing resource scheduling method as described in any one of claims 1 to 6.

Images