Computing power management method and device, equipment, medium and computer program product

Abstract

This application provides a computing power management method, apparatus, device, medium, and computer program product. The method includes: virtualizing computing resources to obtain a dynamic resource pool; obtaining a computing power scheduling strategy for a task queue and the resources required for each task to be processed in the task queue; integrating computing power fragments in the dynamic resource pool based on the computing power scheduling strategy and the resources required for each task to be processed to obtain a computing power scheduling scheme; and scheduling the task queue based on the computing power scheduling scheme. The computing power management method provided by this application mainly includes a computing virtualization part and a heterogeneous computing power scheduling part. In the computing virtualization part, this application achieves fine-grained partitioning of computing power resources by uniformly virtualizing computing resources; in the heterogeneous computing power scheduling part, this application reduces resource fragmentation generated during the allocation of computing power resources and improves the utilization rate of computing power resources through a more flexible computing power scheduling strategy.

Description

Technical Field

[0001] This application relates to the field of computing power resource management technology, and in particular to a computing power management method, apparatus, equipment, medium and computer program product. Background Technology

[0002] In edge computing scenarios, there is a need to deploy multiple small model inference tasks simultaneously. A single task may not occupy the entire computing power card, thus avoiding significant resource waste. Therefore, it is necessary to virtualize a single computing device for multiple tasks. For dedicated computing cards, such as Graphics Processing Units (GPUs), virtualization based on virtual machines virtualizes the GPU hardware device for use by multiple kernel-based Virtual Machines (KVMs). Each KVM has its own driver installed, ensuring both full GPU functionality within the KVM and isolation and sharing of GPU resources. To improve the utilization of computing resources, it is necessary to purchase corresponding software licenses, pre-divide the computing card into different fine-grained segments, and then mount the segmented virtual devices to the KVMs. Therefore, a lightweight management method for heterogeneous computing power virtualization at the edge is urgently needed to improve platform operating efficiency and reduce maintenance costs. Summary of the Invention

[0003] This application provides a computing power management method, apparatus, device, medium, and computer program product to solve the problems that existing virtualization methods based on virtual machines are not flexible enough in terms of virtualization of computing resources and have low utilization of computing resources.

[0004] This application provides a computing power management method, including the following steps: The computing resources are virtualized to obtain a dynamic resource pool; Obtain the computing power scheduling strategy of the task queue, as well as the resources required for each task to be processed in the task queue; Based on the computing power scheduling strategy and the resources required for each task to be processed, the computing power fragments of the dynamic resource pool are integrated to obtain a computing power scheduling scheme. The task queue is scheduled based on the aforementioned computing power scheduling scheme.

[0005] According to the computing power management method provided in this application, the computing power resources include central processing unit (CPU) resources, image processing unit (GPU) resources, and neural network processing unit (NPU) resources; the virtualization process of the computing power resources to obtain a dynamic resource pool includes: The application Kubernetes, which is based on containerized management, virtualizes the CPU resources to obtain CPU virtualization resources. An API interception scheme based on the CUDA unified computing device architecture is used to virtualize the GPU resources to obtain GPU virtualization resources. The NPU resources are virtualized and partitioned to obtain NPU virtualized resources; A dynamic resource pool is constructed based on the CPU virtualization resources, the GPU virtualization resources, and the NPU virtualization resources.

[0006] According to the computing power management method provided in this application, the computing power scheduling strategy includes a priority scheduling strategy; the step of integrating computing power fragments in the dynamic resource pool based on the computing power scheduling strategy and the resources required for each task to be processed, to obtain a computing power scheduling scheme, includes: Get the priority of each pending task; When the dynamic resource pool meets the resource requirements of multiple pending tasks, the multiple pending tasks are scheduled in descending order of priority to obtain a computing power scheduling scheme. If the dynamic resource pool does not meet the resource requirements of multiple pending tasks, the computing power fragments are released in order of priority from low to high. The computing power fragments are integrated to obtain a computing power scheduling scheme.

[0007] According to the computing power management method provided in this application, the computing power scheduling strategy further includes an affinity scheduling strategy, an anti-affinity scheduling strategy, an exclusive scheduling strategy, and a preemptive scheduling strategy; the step of integrating computing power fragments in the dynamic resource pool based on the computing power scheduling strategy and the resources required for each task to be processed to obtain a computing power scheduling scheme further includes: When the computing power scheduling strategy is the affinity scheduling strategy, a label is added to each task to be processed; when the computing power scheduling strategy is the anti-affinity scheduling strategy, a label is added to the target node; the target node is the node that processes the task. The computing power scheduling scheme is determined based on the added tags; When the computing power scheduling strategy is the exclusive scheduling strategy, the computing power scheduling scheme is determined to be task-exclusive computing power resources; If the computing power scheduling strategy is the preemptive exclusive scheduling strategy and the dynamic resource pool does not meet the requirements of the first task, the second task shall be suspended or terminated. Provide the computing resources released by the second task for the first task; the first task has a higher priority than the second task.

[0008] According to the computing power management method provided in this application, the computing power scheduling strategy further includes a binning scheduling strategy and an idle scheduling strategy; the step of integrating computing power fragments in the dynamic resource pool based on the computing power scheduling strategy and the resources required for each task to be processed, to obtain a computing power scheduling scheme, further includes: When the computing power scheduling strategy is the bin packing scheduling strategy, the computing power scheduling scheme is determined based on the resource utilization of each task processing node; When the computing power scheduling strategy is the idle scheduling strategy, the computing power scheduling scheme is determined based on load balancing; the load balancing is determined based on the task scheduling information of each task processing node.

[0009] According to the computing power management method provided in this application, the step of scheduling the task queue based on the computing power scheduling scheme includes: Obtain the operation information of the computing resources and the task to be processed, as well as the usage information of the computing resources; The computing resources are monitored based on the operational information and the usage information.

[0010] This application also provides a computing power management device, including the following modules: The computing resource virtualization module is used to virtualize computing resources to obtain a dynamic resource pool; The acquisition module is used to acquire the computing power scheduling strategy of the task queue, as well as the resources required for each task to be processed in the task queue. The computing power fragment integration module is used to integrate computing power fragments in the dynamic resource pool based on the computing power scheduling strategy and the resources required for each task to be processed, so as to obtain a computing power scheduling scheme. The computing power task scheduling module is used to schedule the task queue based on the computing power scheduling scheme.

[0011] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement any of the computing power management methods described above.

[0012] This application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the computing power management method as described above.

[0013] This application also provides a computer program product, including a computer program that, when executed by a processor, implements any of the computing power management methods described above.

[0014] The computing power management method, apparatus, equipment, media, and computer program products provided in this application mainly include a computing virtualization part and a heterogeneous computing power scheduling part. In the computing virtualization part, this application realizes fine-grained partitioning of computing power resources by performing unified virtualization processing on computing power resources. In the heterogeneous computing power scheduling part, this application reduces resource fragmentation generated during the allocation of computing power resources and improves the utilization rate of computing power resources through a more flexible computing power scheduling strategy. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is one of the flowcharts of the computing power management method provided in this application.

[0017] Figure 2 This is the second flowchart of the computing power management method provided in this application.

[0018] Figure 3 This is a schematic diagram of the computing power management device provided in this application.

[0019] Figure 4 This is a schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] The following is combined Figures 1-4 This application describes the computing power management method, apparatus, equipment, media, and computer program product.

[0022] Figure 1 This is one of the flowcharts illustrating the computing power management method provided in this application, such as... Figure 1 As shown, the method includes the following: Step 100: Virtualize the computing resources to obtain a dynamic resource pool; Specifically, this application provides a computing power management method for a heterogeneous computing power virtualization management system at the edge. The heterogeneous computing power virtualization management system at the edge mainly includes a computing virtualization module, a heterogeneous computing power scheduling module, and an operation and maintenance module.

[0023] The computing virtualization module is used to virtualize computing resources. It mainly includes a Central Processing Unit (CPU) virtualization submodule, a GPU virtualization submodule, and a Neural Network Processing Unit (NPU) virtualization submodule, which are used to virtualize the CPU, GPU, and NPU computing chips respectively, building a dynamic resource pool.

[0024] Step 200: Obtain the computing power scheduling strategy of the task queue, and the resources required for each task to be processed in the task queue; Specifically, in the heterogeneous computing power scheduling module, this application proposes a variety of scheduling strategies applicable to virtualized computing power. Based on the resources required by each task to be processed in the task queue and the importance of each task to be processed in the computing power scheduling strategy, the computing power resources in the dynamic resource pool are scheduled to complete the tasks to be processed.

[0025] Step 300: Based on the computing power scheduling strategy and the resources required for each task to be processed, the computing power fragments of the dynamic resource pool are integrated to obtain a computing power scheduling scheme. Step 400: Schedule the task queue based on the computing power scheduling scheme.

[0026] Specifically, the heterogeneous computing power scheduling module obtains information on currently available computing power resources in the dynamic resource pool through computing power awareness. Based on the type and quantity of requested resources and the selected scheduling strategy, it runs container instances of the corresponding resources on the best-matching nodes to complete the task scheduling of computing power resources.

[0027] The computing power management method provided in this application mainly includes a computing virtualization part and a heterogeneous computing power scheduling part. In the computing virtualization part, this application realizes fine-grained partitioning of computing power resources by performing unified virtualization processing on computing power resources. In the heterogeneous computing power scheduling part, this application reduces resource fragmentation generated during the allocation of computing power resources and improves the utilization rate of computing power resources by adopting a more flexible computing power scheduling strategy.

[0028] In one embodiment, the computing power management method provided in this application may further include: Step 110: Using Kubernetes, a containerized application, the CPU resources are virtualized to obtain virtualized CPU resources. Step 120: Based on the CUDA unified computing device architecture, the application programming interface (API) interception scheme is used to virtualize the GPU resources to obtain GPU virtualization resources; Step 130: Perform virtualization partitioning on the NPU resources to obtain NPU virtualization resources; Step 140: Construct a dynamic resource pool based on the CPU virtualization resources, the GPU virtualization resources, and the NPU virtualization resources.

[0029] Specifically, the computing resource virtualization processing part includes CPU virtualization, GPU virtualization and NPU virtualization, and constructs a dynamic resource pool through virtualized computing resources.

[0030] In CPU virtualization, provided by the Kubernetes system (an open-source system for managing containerized applications across multiple hosts in a cloud platform) and container runtime, container access to the CPU is limited through the cgroup feature of the Linux kernel (an operating system feature used to limit, control, and isolate the resources of a group of processes).

[0031] GPU virtualization can be implemented using a software virtualization scheme based on CUDA (a parallel computing platform and programming model) application programming interface (API) hijacking. After installing the GPU device plugin on a Kubernetes node configured with a GPU, the GPU Manager module (a Kubernetes-based GPU resource management tool) discovers the GPU device and registers the GPU device information with the Kubelet components (worker nodes or compute nodes) on the Kubernetes cluster. The GPU device plugin registers the discovered GPU device as multiple (e.g., 100) AICores (dedicated computing units in the GPU) and MEM (memory) corresponding to the device's memory size, representing fine-grained virtualization and actual resource request and allocation.

[0032] When a user requests virtualized GPU device resources, the Kubelet component sends a resource request to the GPU Manager. The GPU Scheduler then selects a suitable node based on the scheduling policy and creates a virtual GPU, which is allocated to the user's container instance. Simultaneously, the GPU Scheduler modifies the CUDA library files inside the container. When a user initiates a GPU call request within the container, this request is intercepted in user space, and the GPU resources allocated to that request are restricted. The request is then redirected to the host machine's CUDA program and GPU driver, and the result is returned to the user container after execution. Furthermore, this instance's device plugin uses time-division multiplexing to handle concurrent requests from different containers to the same GPU device, achieving GPU sharing.

[0033] Dynamic virtualization is employed in NPU virtualization, unlike static virtualization which requires pre-splitting NPUs into fixed virtual NPU groups based on templates. Dynamic virtualization eliminates the need for users to pre-split virtual NPUs and automatically reclaims them after task completion, effectively supporting scenarios where user computing power demands constantly change. After installing the NPU device plugin on a Kubernetes node configured with NPUs, the component reports the number of available AICores on its machine. When a user requests virtualized NPU device resources, the NPU-Scheduler calculates and schedules the task to a node that meets the requirements. Upon receiving the request, the node's NPU device plugin automatically splits the virtual NPU device and mounts it to the user's container instance, thus completing the entire dynamic virtualization process.

[0034] This embodiment adopts a container-based virtualization approach. By sharing the host operating system kernel, it can provide sufficient isolation with extremely low resource consumption, while greatly improving deployment speed, portability and compatibility, and significantly reducing management complexity.

[0035] Figure 2 This is the second flowchart of the computing power management method provided in this application, as shown below. Figure 2 As shown, the method may further include: Step 301: Obtain the priority of each pending task; Step 302: If the dynamic resource pool meets the resource requirements of multiple tasks to be processed, schedule the multiple tasks to be processed in order of priority from high to low to obtain a computing power scheduling scheme. Step 303: If the dynamic resource pool does not meet the resource requirements of multiple pending tasks, release computing power fragments in order of priority from low to high. Step 304: Integrate the computing power fragments to obtain a computing power scheduling scheme.

[0036] Specifically, in the heterogeneous computing power scheduling section, this application provides various computing power scheduling strategies. Taking the priority scheduling strategy as an example, the priority scheduling strategy sets priorities for different user containers, for example, from 1 to 10, with higher numbers indicating higher priorities. When multiple computing tasks are in the (task) waiting queue, and the computing power resource requests can be satisfied, the scheduler will schedule the execution of the higher-priority computing tasks. When computing power resources are insufficient, for low-priority tasks, if the computing power resources they occupy can satisfy the needs of higher-priority tasks, the scheduler is allowed to pause the low-priority tasks, thereby freeing up computing power resources to satisfy the higher-priority tasks.

[0037] This embodiment obtains information on currently available resources through computing power awareness, and runs container instances of the corresponding resources on the best-matching node based on the type and quantity of the requested resources and the selected priority scheduling strategy.

[0038] In one embodiment, the computing power management method provided in this application may further include: Step 305: When the computing power scheduling strategy is the affinity scheduling strategy, add a label to each task to be processed; when the computing power scheduling strategy is the anti-affinity scheduling strategy, add a label to the target node; the target node is the node that processes the task. Step 306: Determine the computing power scheduling scheme based on the added tags; Step 307: When the computing power scheduling strategy is the exclusive scheduling strategy, determine the computing power scheduling scheme as task exclusive computing power resources; Step 308: If the computing power scheduling strategy is the preemptive exclusive scheduling strategy and the dynamic resource pool does not meet the requirements of the first task, suspend or terminate the second task. Step 309: Provide the computing resources released by the second task to the first task; the first task has a higher priority than the second task.

[0039] Specifically, we take affinity scheduling strategy, anti-affinity scheduling strategy, exclusive scheduling strategy, and preemptive scheduling strategy as examples.

[0040] Affinity scheduling policies allow users to add tags to container tasks when submitting them, thus scheduling the containers to nodes with specific attributes or configurations. Anti-affinity scheduling policies, on the other hand, avoid scheduling containers to nodes with specific tags, achieving isolation between containers or preventing resource contention. Exclusive scheduling policies allow tasks to exclusively occupy allocated computing resources in the virtualization environment, ensuring resource stability; preemptive scheduling policies allow the scheduler to free up resources for higher-priority tasks by terminating or evicting lower-priority tasks when node resources are insufficient.

[0041] This embodiment obtains information on currently available resources through computing power awareness, and runs container instances of the corresponding resources on the best-matching node based on the type and quantity of the requested resources, as well as the selected affinity scheduling strategy, anti-affinity scheduling strategy, exclusive scheduling strategy, and preemptive scheduling strategy.

[0042] In one embodiment, the computing power management method provided in this application may further include: Step 310: When the computing power scheduling strategy is the bin packing scheduling strategy, determine the computing power scheduling scheme based on the resource utilization rate of each task processing node. Step 311: When the computing power scheduling strategy is the idle scheduling strategy, determine the computing power scheduling scheme based on load balancing; the load balancing is determined based on the task scheduling information of each task processing node.

[0043] Specifically, let's take bin packing scheduling strategy and idle scheduling strategy as examples. The bin packing scheduling strategy prioritizes scheduling task containers to nodes with high resource utilization to reduce the fragmentation of idle resources on each node and improve the utilization of cluster resources; the idle scheduling strategy, on the other hand, schedules them evenly across all resources to balance the load.

[0044] The computing power fragmentation consolidation and scheduling strategy is an algorithm strategy determined by this application for the characteristics of heterogeneous computing power virtualization at the edge. It aims to reduce the remaining fragmented computing power of multiple computing cards. The algorithm calculates a scheme for migratable computing power resources and smoothly migrates them without interrupting running tasks. It concentrates fragmented resources on one or more cards, allowing users to deploy more inference services and ultimately improve the utilization rate of graphics card resources.

[0045] After the system has been running for a long time, tasks are constantly starting and stopping. The amount of remaining resources on each computing device is different, resulting in computing fragments of different sizes. At this time, when a new task needs to be scheduled but computing resources are insufficient, and computing fragment consolidation is enabled, the scheduler will traverse tasks with a priority of less than 5 (i.e., low-priority tasks), attempt to consolidate resource allocation, calculate the maximum amount of remaining resources in computing card j after these tasks are migrated from the original computing card j to computing card i, and when this amount is greater than the amount of remaining resources in computing card i, the task is hot-migrated to computing card i. This cycle is repeated until the end, so that computing fragments are gathered on the same computing card as much as possible, and tasks are rescheduled according to the obtained scheme.

[0046] This embodiment integrates and schedules computing power fragments, allowing them to be clustered on the same computing power card as much as possible, thereby rescheduling tasks.

[0047] In one embodiment, the computing power management method provided in this application may further include: Step 500: Obtain the operation information of the computing resources and the task to be processed, as well as the usage information of the computing resources; Step 600: Monitor the computing resources based on the operation information and the usage information.

[0048] Specifically, the operations and maintenance (O&M) module includes hardware monitoring, software monitoring, and alarms and maintenance of corresponding resource metrics. Considering the limited resources of edge computing platforms, this application provides a lightweight yet comprehensive O&M monitoring solution. It collects the overall platform hardware operation status and resource usage in all task containers through Exporter (a program capable of collecting metric data from target systems or services); it collects task operation status, including task logs, operation logs, and API call statistics, through Promtail (a lightweight log collection agent) and Loki (a log storage and query engine); and it configures alarm policies for hardware resources, business resources, and log information through Alert Manager (the core component of the Loki log alarm system), automatically sending corresponding alarms based on the collected information. This enables more advanced O&M functions, such as elastic scaling of inference services, elastic scaling, and container upgrades / downgrades. The O&M monitoring module in this case can achieve on-site monitoring of business and node performance.

[0049] This embodiment uses an operations and maintenance module, which only requires the deployment of one set of operations and maintenance monitoring components to achieve real-time monitoring of all software and hardware, and is compatible with a variety of alarm indicators.

[0050] The computing power management device provided in this application is described below. The computing power management device described below can be referred to in correspondence with the computing power management method described above.

[0051] Please refer to Figure 3 This application also provides a computing power management device, including: The computing resource virtualization module 401 is used to virtualize computing resources to obtain a dynamic resource pool; The acquisition module 402 is used to acquire the computing power scheduling strategy of the task queue, as well as the resources required for each task to be processed in the task queue. The computing power fragment integration module 403 is used to integrate computing power fragments in the dynamic resource pool based on the computing power scheduling strategy and the resources required for each task to be processed, so as to obtain a computing power scheduling scheme. The computing power task scheduling module 404 is used to schedule the task queue based on the computing power scheduling scheme.

[0052] Optionally, the computing resources include central processing unit (CPU) resources, image processing unit (GPU) resources, and neural network processing unit (NPU) resources; the computing resources virtualization module includes: The CPU resource virtualization processing unit is used to virtualize the CPU resources based on the containerized application Kubernetes to obtain CPU virtualized resources. The GPU resource virtualization processing unit is used to perform virtualization processing on the GPU resources based on the CUDA unified computing device architecture API interception scheme to obtain GPU virtualization resources. The NPU resource virtualization processing unit is used to virtualize and divide the NPU resources to obtain NPU virtualized resources. The dynamic resource pool construction unit is used to construct a dynamic resource pool based on the CPU virtualization resources, the GPU virtualization resources, and the NPU virtualization resources.

[0053] Optionally, the computing power scheduling strategy includes a priority scheduling strategy; the computing power fragmentation integration module includes: Priority acquisition unit, used to acquire the priority of each task to be processed; The task scheduling unit is used to schedule multiple tasks in descending order of priority, in order of priority, to obtain a computing power scheduling scheme, provided that the dynamic resource pool meets the resource requirements of multiple tasks. The computing power fragment release unit is used to release computing power fragments in order of priority from low to high when the dynamic resource pool does not meet the resource requirements of multiple pending tasks. The computing power fragment integration unit is used to integrate the computing power fragments to obtain a computing power scheduling scheme.

[0054] Optionally, the computing power scheduling strategy further includes an affinity scheduling strategy, an anti-affinity scheduling strategy, an exclusive scheduling strategy, and a preemptive scheduling strategy; the computing power fragmentation integration module further includes: The tag adding unit is used to add a tag to each task to be processed when the computing power scheduling strategy is the affinity scheduling strategy; and to add a tag to the target node when the computing power scheduling strategy is the anti-affinity scheduling strategy; wherein the target node is the node that processes the task. The first computing power scheduling scheme determination unit is used to determine the computing power scheduling scheme based on the added tags; The second computing power scheduling scheme determination unit is used to determine the computing power scheduling scheme as task-exclusive computing power resources when the computing power scheduling strategy is the exclusive scheduling strategy. The task processing unit is used to suspend or terminate the second task when the computing power scheduling strategy is the preemptive exclusive scheduling strategy and the dynamic resource pool does not meet the requirements of the first task. A computing resource providing unit is used to provide the computing resources released by the second task to the first task; the first task has a higher priority than the second task.

[0055] Optionally, the computing power scheduling strategy further includes a binning scheduling strategy and an idle scheduling strategy; the computing power fragment integration module further includes: The third computing power scheduling scheme determination unit is used to determine the computing power scheduling scheme based on the resource utilization rate of each task processing node when the computing power scheduling strategy is the binning scheduling strategy. The fourth computing power scheduling scheme determination unit is used to determine a computing power scheduling scheme based on load balancing when the computing power scheduling strategy is the idle scheduling strategy; the load balancing is determined based on the task scheduling information of each task processing node.

[0056] Optionally, the computing power management device further includes: The information acquisition module is used to acquire the operation information of the computing resources and the task to be processed, as well as the usage information of the computing resources; The computing resource monitoring module is used to monitor the computing resources based on the operation information and the usage information.

[0057] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communications interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a computing power management method, which includes: virtualizing computing resources to obtain a dynamic resource pool; obtaining the computing power scheduling policy of the task queue and the required resources of each task to be processed in the task queue; integrating computing power fragments in the dynamic resource pool based on the computing power scheduling policy and the required resources of each task to be processed to obtain a computing power scheduling scheme; and scheduling the task queue based on the computing power scheduling scheme.

[0058] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0059] On the other hand, this application also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the computing power management method provided by the above methods. The method includes: virtualizing computing power resources to obtain a dynamic resource pool; obtaining a computing power scheduling strategy for a task queue and the resources required for each task to be processed in the task queue; integrating computing power fragments in the dynamic resource pool based on the computing power scheduling strategy and the resources required for each task to be processed to obtain a computing power scheduling scheme; and scheduling the task queue based on the computing power scheduling scheme.

[0060] In another aspect, this application also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program is implemented to perform the computing power management methods provided by the above methods. The method includes: virtualizing computing resources to obtain a dynamic resource pool; obtaining a computing power scheduling policy for a task queue and the resources required for each task to be processed in the task queue; integrating computing power fragments in the dynamic resource pool based on the computing power scheduling policy and the resources required for each task to be processed to obtain a computing power scheduling scheme; and scheduling the task queue based on the computing power scheduling scheme.

[0061] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0062] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0063] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A computing power management method, characterized in that, include: The computing resources are virtualized to obtain a dynamic resource pool; Obtain the computing power scheduling strategy of the task queue, as well as the resources required for each task to be processed in the task queue; Based on the computing power scheduling strategy and the resources required for each task to be processed, the computing power fragments of the dynamic resource pool are integrated to obtain a computing power scheduling scheme. The task queue is scheduled based on the aforementioned computing power scheduling scheme.

2. The computing power management method according to claim 1, characterized in that, The computing resources include central processing unit (CPU) resources, image processing unit (GPU) resources, and neural network processing unit (NPU) resources; the virtualization of the computing resources to obtain a dynamic resource pool includes: The application Kubernetes, which is based on containerized management, virtualizes the CPU resources to obtain CPU virtualization resources. An API interception scheme based on the CUDA unified computing device architecture is used to virtualize the GPU resources to obtain GPU virtualization resources. The NPU resources are virtualized and partitioned to obtain NPU virtualized resources; A dynamic resource pool is constructed based on the CPU virtualization resources, the GPU virtualization resources, and the NPU virtualization resources.

3. The computing power management method according to claim 1, characterized in that, The computing power scheduling strategy includes a priority scheduling strategy; the computing power scheduling scheme is obtained by integrating computing power fragments in the dynamic resource pool based on the computing power scheduling strategy and the resources required for each task to be processed, including: Get the priority of each pending task; When the dynamic resource pool meets the resource requirements of multiple pending tasks, the multiple pending tasks are scheduled in descending order of priority to obtain a computing power scheduling scheme. If the dynamic resource pool does not meet the resource requirements of multiple pending tasks, the computing power fragments are released in order of priority from low to high. The computing power fragments are integrated to obtain a computing power scheduling scheme.

4. The computing power management method according to claim 1, characterized in that, The computing power scheduling strategy further includes affinity scheduling strategy, anti-affinity scheduling strategy, exclusive scheduling strategy, and preemptive scheduling strategy; the step of integrating computing power fragments in the dynamic resource pool based on the computing power scheduling strategy and the resources required for each task to be processed to obtain a computing power scheduling scheme also includes: When the computing power scheduling strategy is the affinity scheduling strategy, a label is added to each task to be processed; when the computing power scheduling strategy is the anti-affinity scheduling strategy, a label is added to the target node; the target node is the node that processes the task. The computing power scheduling scheme is determined based on the added tags; When the computing power scheduling strategy is the exclusive scheduling strategy, the computing power scheduling scheme is determined to be task-exclusive computing power resources; If the computing power scheduling strategy is the preemptive exclusive scheduling strategy and the dynamic resource pool does not meet the requirements of the first task, the second task shall be suspended or terminated. Provide the computing resources released by the second task for the first task; the first task has a higher priority than the second task.

5. The computing power management method according to claim 1, characterized in that, The computing power scheduling strategy further includes a binning scheduling strategy and an idle scheduling strategy; the step of integrating computing power fragments in the dynamic resource pool based on the computing power scheduling strategy and the resources required for each task to be processed, to obtain a computing power scheduling scheme, further includes: When the computing power scheduling strategy is the bin packing scheduling strategy, the computing power scheduling scheme is determined based on the resource utilization of each task processing node; When the computing power scheduling strategy is the idle scheduling strategy, the computing power scheduling scheme is determined based on load balancing; the load balancing is determined based on the task scheduling information of each task processing node.

6. The computing power management method according to claim 1, characterized in that, The process of scheduling the task queue based on the computing power scheduling scheme then includes: Obtain the operation information of the computing resources and the task to be processed, as well as the usage information of the computing resources; The computing resources are monitored based on the operational information and the usage information.

7. A computing power management device, characterized in that, include: The computing resource virtualization module is used to virtualize computing resources to obtain a dynamic resource pool; The acquisition module is used to acquire the computing power scheduling strategy of the task queue, as well as the resources required for each task to be processed in the task queue. The computing power fragment integration module is used to integrate computing power fragments in the dynamic resource pool based on the computing power scheduling strategy and the resources required for each task to be processed, so as to obtain a computing power scheduling scheme. The computing power task scheduling module is used to schedule the task queue based on the computing power scheduling scheme.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the computing power management method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the computing power management method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the computing power management method as described in any one of claims 1 to 6.

Images