GPU resource scheduling method, device, electronic device and storage medium

Abstract

An embodiment of the present invention provides a GPU resource scheduling method, apparatus, electronic device, and storage medium. The method includes: obtaining a task to be processed; allocating target computing resources from the resource pool to the task to be processed based on the target processing algorithm corresponding to the task to be processed and the idle status of computing resources in the resource pool; the computing resources include physical GPUs and virtualized GPUs, and the virtualized GPUs include physical layer virtualized GPUs and application layer virtualized GPUs. The electronic device can allocate target computing resources to the task to be processed from the resource pool without having to rely heavily on the virtualization of the underlying GPU computing resources. Even if the underlying GPU of the computing resources does not support physical layer virtualization, it can combine application layer algorithm scheduling with underlying virtualization technology to enable a single GPU to run multiple algorithms simultaneously, greatly reducing the limitations of GPU resource scheduling and enabling flexible scheduling of GPU resources.

Description

Technical Field

[0001] The present invention relates to the technical field of resource scheduling, and in particular to a GPU resource scheduling method, device, electronic device and storage medium. Background Art

[0002] A graphics processing unit (GPU) is a microprocessor specialized for image and graphics-related computing. Security systems often need to analyze and process large amounts of image and video information, making GPUs crucial for security applications.

[0003] In actual image processing applications, multiple GPUs are typically used to handle computing tasks, and a scheduling system is used to plan and manage the GPUs to fully utilize their computing power. Common scheduling systems typically use a single GPU as the minimum management granularity.

[0004] Current image processing tasks require a wide variety of intelligent algorithms, while the number of GPUs is relatively small, and fragmented analysis is often required. To fully utilize GPU computing power, underlying virtualization technology can be used to virtualize a single physical GPU into multiple GPU resources, allowing a single GPU to run multiple intelligent algorithms simultaneously. However, this solution relies heavily on the virtualization of underlying GPU computing resources, resulting in significant limitations and inflexibility in GPU resource scheduling. Summary of the Invention

[0005] The purpose of the embodiments of the present invention is to provide a GPU resource scheduling method, apparatus, electronic device, and storage medium to achieve flexible GPU resource scheduling without relying on underlying GPU computing resource virtualization. The specific technical solution is as follows:

[0006] In a first aspect, an embodiment of the present invention provides a GPU resource scheduling method, the method comprising:

[0007] Get pending tasks;

[0008] According to the target processing algorithm corresponding to the task to be processed and the idle status of computing resources in the resource pool, target computing resources are allocated to the task to be processed from the resource pool; wherein, the computing resources include physical GPUs and virtualized GPUs, and the virtualized GPUs include physical layer virtualized GPUs and application layer virtualized GPUs.

[0009] Optionally, before the step of obtaining the task to be processed, the method further includes:

[0010] The GPUs in the resource pool are virtualized to obtain the computing resources, wherein the resource pool includes a plurality of preset GPUs participating in resource scheduling; the proportion of the virtualized GPU in the computing resources of the resource pool satisfies a first ratio; and / or the proportion of the virtualized GPU of the target computing node in the resource pool in the computing resources of the target computing node satisfies a second ratio.

[0011] Optionally, the step of allocating target computing resources to the task to be processed from the computing resources according to the target processing algorithm corresponding to the task to be processed and the idle status of computing resources in the resource pool includes:

[0012] Determining whether the target processing algorithm can be executed by a virtualized GPU;

[0013] If the target processing algorithm can be executed by a virtualized GPU, determining whether there is an idle virtualized GPU in the resource pool that meets the scheduling requirements;

[0014] Determining a target virtualized GPU from the idle virtualized GPUs that meet the scheduling requirements, wherein the target virtualized GPU includes a first idle physical layer virtualized GPU or a first idle application layer virtualized GPU;

[0015] Send the pending task to the target virtualized GPU.

[0016] Optionally, the step of determining whether the target processing algorithm can be executed by a virtualized GPU includes:

[0017] Determining whether the processing algorithms corresponding to the virtualization technology pre-configured on the virtualized GPU include the target processing algorithm;

[0018] If included, it is determined that the target processing algorithm can be executed by the virtualized GPU.

[0019] Optionally, the step of determining whether there is an idle virtualized GPU in the resource pool that meets the scheduling requirements includes:

[0020] Determining whether there is an idle virtualized GPU in the resource pool based on the virtualization ratio of the virtualized GPU in the resource pool and the occupied computing power of the virtualized GPU;

[0021] If the idle virtualized GPU exists, determining whether the remaining computing power of the idle virtualized GPU meets the computing power requirement of the task to be processed;

[0022] If so, it is determined that the idle virtualized GPU is the idle virtualized GPU that meets the scheduling requirement.

[0023] Optionally, the step of determining a target virtualized GPU from the idle virtualized GPUs that meet the scheduling requirements includes:

[0024] Determining whether there is a first virtualized GPU that supports a target physical layer virtualization technology among the idle virtualized GPUs that meet the scheduling requirements, wherein the target physical layer virtualization technology is a physical layer virtualization technology applicable to the target processing algorithm;

[0025] If there are multiple first virtualized GPUs, a first idle physical layer virtualization GPU is determined from the first virtualized GPUs based on the priority of the physical layer virtualization technology corresponding to each first virtualized GPU, wherein the priority of the physical layer virtualization technology is determined based on the processing performance of the simulated virtualized GPU corresponding to the first virtualized GPU, and the simulated virtualized GPU is obtained by virtualizing the GPU corresponding to the first virtualized GPU through multiple physical layer virtualization technologies.

[0026] Optionally, the step of determining the target virtualized GPU from the idle VPUs that meet the scheduling requirements further includes:

[0027] Determining whether there is a second virtualized GPU that supports a target application layer virtualization technology among the idle virtualized GPUs that meet the scheduling requirements, wherein the target application layer virtualization technology is an application layer virtualization technology applicable to the target processing algorithm;

[0028] For a computing node having the second virtualized GPU, determine whether the second virtualized GPU corresponding to the computing node is running other processing algorithms pre-bound and associated with the target processing algorithm; if so, determine that the second virtualized GPU corresponding to the computing node is the first idle application layer virtualized GPU; if not, determine whether the second virtualized GPU corresponding to the next computing node is running other processing algorithms pre-bound and associated with the target processing algorithm; until it is determined that all second virtualized GPUs are no longer running other processing algorithms pre-bound and associated with the target processing algorithm, determine the second virtualized GPU that supports the target processing algorithm as the first idle application layer virtualized GPU; or,

[0029] If there are multiple second virtualized GPUs in the resource pool, for each second virtualized GPU, determine whether the second virtualized GPU is running other processing algorithms that are pre-bound and associated with the target processing algorithm; if so, determine that the second virtualized GPU is the first idle application layer virtualized GPU; if not, determine that the second virtualized GPU that supports the target processing algorithm is the first idle application layer virtualized GPU.

[0030] In a second aspect, an embodiment of the present invention provides a GPU resource scheduling device, the device comprising:

[0031] A pending task acquisition module is used to obtain pending tasks;

[0032] A resource allocation module is used to allocate target computing resources from the resource pool to the task to be processed based on the target processing algorithm corresponding to the task to be processed and the idle status of computing resources in the resource pool; wherein, the computing resources include physical GPUs and virtualized GPUs, and the virtualized GPUs include physical layer virtualized GPUs and application layer virtualized GPUs.

[0033] Optionally, the device further includes:

[0034] A virtualization processing module is used to virtualize the GPU in the resource pool before obtaining the task to be processed to obtain the computing resources, wherein the resource pool includes a plurality of preset GPUs participating in resource scheduling; the proportion of the virtualized GPU in the computing resources of the resource pool meets a first ratio; and / or the proportion of the virtualized GPU of the target computing node in the resource pool in the computing resources of the target computing node meets a second ratio.

[0035] Optionally, the resource allocation module includes:

[0036] A first determining unit, configured to determine whether the target processing algorithm can be executed by a virtualized GPU;

[0037] a second determining unit, configured to determine whether there is an idle virtualized GPU in the resource pool that meets the scheduling requirements if the target processing algorithm can be executed by the virtualized GPU;

[0038] a target GPU determining unit, configured to determine a target virtualized GPU from the idle virtualized GPUs that meet the scheduling requirements, wherein the target virtualized GPU includes a first idle physical layer virtualized GPU or a first idle application layer virtualized GPU;

[0039] The task sending unit is used to send the to-be-processed task to the target virtualized GPU.

[0040] Optionally, the first judgment unit includes:

[0041] an algorithm determination subunit, configured to determine whether the processing algorithms corresponding to the virtualization technology pre-configured on the virtualized GPU include the target processing algorithm;

[0042] The determining subunit is configured to determine whether the target processing algorithm can be executed by the virtualized GPU if the target processing algorithm is included.

[0043] Optionally, the second judgment unit includes:

[0044] an idle GPU determination subunit, configured to determine whether there is an idle virtualized GPU in the resource pool based on a virtualization ratio of the virtualized GPUs in the resource pool and an occupied computing power of the virtualized GPUs;

[0045] A computing power requirement determination subunit, configured to determine, if the idle virtualized GPU exists, whether the remaining computing power of the idle virtualized GPU meets the computing power requirement of the task to be processed;

[0046] The idle GPU determining subunit is configured to determine that the idle virtualized GPU is the idle virtualized GPU that meets the scheduling requirement if the computing power requirement of the task to be processed is met.

[0047] Optionally, the target GPU determination unit includes:

[0048] a first virtualized GPU determination subunit, configured to determine whether there is a first virtualized GPU that supports a target physical layer virtualization technology among the idle virtualized GPUs that meet the scheduling requirements, wherein the target physical layer virtualization technology is a physical layer virtualization technology applicable to the target processing algorithm;

[0049] The first target GPU determination subunit is used to determine a first idle physical layer virtualization GPU from the first virtualized GPUs based on the priority of the physical layer virtualization technology corresponding to each first virtualized GPU if there are multiple first virtualized GPUs, wherein the priority of the physical layer virtualization technology is determined based on the processing performance of the simulated virtualized GPU corresponding to the first virtualized GPU, and the simulated virtualized GPU is obtained by virtualizing the GPU corresponding to the first virtualized GPU through multiple physical layer virtualization technologies.

[0050] Optionally, the target GPU determination unit further includes:

[0051] a second virtualized GPU determining subunit, configured to determine whether there is a second virtualized GPU that supports a target application layer virtualization technology among the idle virtualized GPUs that meet the scheduling requirements, wherein the target application layer virtualization technology is an application layer virtualization technology applicable to the target processing algorithm;

[0052] The first target GPU determination subunit is configured to determine, for a computing node having the second virtualized GPU, whether the second virtualized GPU corresponding to the computing node is running other processing algorithms pre-bound and associated with the target processing algorithm; if so, determine that the second virtualized GPU corresponding to the computing node is the first idle application layer virtualized GPU; if not, determine whether the second virtualized GPU corresponding to the next computing node is running other processing algorithms pre-bound and associated with the target processing algorithm; until it is determined that all second virtualized GPUs are no longer running other processing algorithms pre-bound and associated with the target processing algorithm, determine the second virtualized GPU that supports the target processing algorithm as the first idle application layer virtualized GPU; or,

[0053] Used to, if there are multiple second virtualized GPUs in the resource pool, determine, for each second virtualized GPU, whether the second virtualized GPU is running other processing algorithms that are pre-bound and associated with the target processing algorithm; if so, determine that the second virtualized GPU is the first idle application layer virtualized GPU; if not, determine that the second virtualized GPU that supports the target processing algorithm is the first idle application layer virtualized GPU.

[0054] In a third aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0055] Memory for storing computer programs;

[0056] The processor is configured to implement any of the method steps described in the first aspect when executing a program stored in the memory.

[0057] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of the first aspects are implemented.

[0058] Beneficial effects of the embodiments of the present invention:

[0059] In the solution provided by the embodiment of the present invention, an electronic device can obtain a task to be processed, and then, based on the target processing algorithm corresponding to the task to be processed and the idle status of computing resources in the resource pool, allocate target computing resources from the resource pool to the task to be processed. The computing resources include physical GPUs and virtualized GPUs, and the virtualized GPUs include physical layer virtualized GPUs and application layer virtualized GPUs. In the above solution, the computing resources include physical GPUs and virtualized GPUs, and the virtualized GPUs include physical layer virtualized GPUs and application layer virtualized GPUs. Therefore, the electronic device can allocate target computing resources from the resource pool to the task to be processed based on the target processing algorithm corresponding to the task to be processed and the idle status of physical GPUs, physical layer virtualized GPUs, and application layer virtualized GPUs in the resource pool. This does not require strong reliance on the virtualization of underlying GPU computing resources. Even if the underlying GPU of the computing resource does not support physical layer virtualization, it can combine application layer algorithm scheduling with underlying virtualization technology to enable a single GPU to run multiple algorithms simultaneously, greatly reducing the limitations of GPU resource scheduling and enabling flexible scheduling of GPU resources. Of course, implementing any product or method of the present invention does not necessarily require achieving all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0061] Figure 1 A flowchart of a GPU resource scheduling method provided by an embodiment of the present invention;

[0062] Figure 2 for Figure 1 A specific flow chart of step S102 in the embodiment shown;

[0063] Figure 3 for Figure 2 A specific flow chart of step S201 in the embodiment shown;

[0064] Figure 4 Based on Figure 3 A flow chart of a method for setting the priority of the physical layer virtualization technology of the illustrated embodiment;

[0065] Figure 5 for Figure 2 A specific flow chart of step S203 in the embodiment shown;

[0066] Figure 6 Based on Figure 5A flow chart of the binding method of the processing algorithm of the illustrated embodiment;

[0067] Figure 7 Based on Figure 1 A schematic diagram of the GPU resource scheduling process of the illustrated embodiment;

[0068] Figure 8 Based on Figure 1 A schematic structural diagram of a GPU resource scheduling system according to the illustrated embodiment;

[0069] Figure 9 Based on Figure 1 A flow chart of a GPU virtualization configuration method according to the illustrated embodiment;

[0070] Figure 10 Based on Figure 1 A flow chart of a GPU resource scheduling method according to the illustrated embodiment;

[0071] Figure 11 A schematic diagram of the structure of a GPU resource scheduling device provided by an embodiment of the present invention;

[0072] Figure 12 The present invention provides a schematic structural diagram of an electronic device. DETAILED DESCRIPTION

[0073] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field based on the present invention are within the scope of protection of the present invention.

[0074] To reduce the limitations of GPU resource scheduling and enable flexible GPU resource scheduling without relying heavily on underlying GPU computing resource virtualization when scheduling GPU resources, embodiments of the present invention provide a GPU resource scheduling method, apparatus, electronic device, computer-readable storage medium, and computer program product. The following first introduces a GPU resource scheduling method provided by an embodiment of the present invention.

[0075] The GPU resource scheduling method provided in the embodiments of the present invention can be applied to any electronic device that requires GPU resource scheduling, including servers and processors in a GPU resource scheduling system, processing devices and control devices used for resource scheduling in an image processing system, and the like, without specific limitation herein. For clarity, these devices are referred to as electronic devices below.

[0076] like Figure 1As shown, a GPU resource scheduling method includes:

[0077] S101, obtaining tasks to be processed.

[0078] S102 , allocating target computing resources from the resource pool to the task to be processed according to the target processing algorithm corresponding to the task to be processed and the idleness of computing resources in the resource pool.

[0079] The computing resources include physical GPUs and virtualized GPUs, and the virtualized GPUs include physical layer virtualized GPUs and application layer virtualized GPUs.

[0080] It can be seen that in the solution provided by the embodiment of the present invention, the electronic device can obtain the task to be processed, and then, according to the target processing algorithm corresponding to the task to be processed and the idle status of the computing resources in the resource pool, the target computing resources can be allocated from the resource pool to the task to be processed. Among them, the computing resources include physical GPUs and virtualized GPUs, and the virtualized GPUs include physical layer virtualized GPUs and application layer virtualized GPUs. In the above solution, the computing resources include physical GPUs and virtualized GPUs, and the virtualized GPUs include physical layer virtualized GPUs and application layer virtualized GPUs. Therefore, the electronic device can allocate the target computing resources from the resource pool to the task to be processed according to the target processing algorithm corresponding to the task to be processed and the idle status of the physical GPUs, physical layer virtualized GPUs, and application layer virtualized GPUs in the resource pool. It does not have to rely heavily on the virtualization of the underlying GPU computing resources. Even if the underlying GPU of the computing resources does not support physical layer virtualization, it can combine the algorithm scheduling of the application layer with the underlying virtualization technology to achieve the simultaneous operation of multiple algorithms on a single GPU, greatly reducing the limitations of GPU resource scheduling and enabling flexible scheduling of GPU resources.

[0081] When image processing is required, in step S101 of the embodiment of the present invention, the electronic device may obtain a pending task. The pending task is an image processing task that needs to be processed, such as a target behavior analysis task, an image recognition task, an image feature analysis task, an image classification task, a vehicle identification task, etc. in a video surveillance system, and is not specifically limited here.

[0082] In the embodiments of the present invention, an electronic device can periodically retrieve pending tasks based on a preset interval. Alternatively, the electronic device can receive an external task processing request and, in response to the task processing request, retrieve pending tasks. In one embodiment, pending tasks can be stored in a cache queue, and the electronic device can retrieve pending tasks from the cache queue at each preset interval.

[0083] Among them, the preset interval duration can be set according to conditions such as the GPU resource scheduling policy and the time required for task processing, and can be 5 seconds, 10 seconds, 15 seconds, etc., which are not specifically limited here. When the electronic device completes the GPU resource scheduling based on the pending task, that is, after the pending task is sent to a certain GPU resource, it can be removed from the cache queue. The electronic device can also receive an external task processing request, and then obtain the pending task corresponding to the task processing request, and perform GPU resource scheduling based on the pending task.

[0084] Image processing tasks can include multiple tasks, including pending tasks, and therefore typically require multiple processing algorithms. Some or all of these algorithms may be suitable for GPU virtualization technology. After a physical GPU is virtualized using virtualization technology to create a virtualized GPU, processing algorithms that are suitable for that virtualization technology can run on the corresponding virtualized GPU, while processing algorithms that are not suitable for that virtualization technology cannot run on the corresponding virtualized GPU.

[0085] After receiving a pending task, the electronic device can determine the target processing algorithm corresponding to the pending task and the availability of computing resources in the resource pool, and then allocate target computing resources from the resource pool to the pending task. The target processing algorithm is the algorithm required to process the pending task. Computing resources can include physical GPUs and virtualized GPUs. Virtualized GPUs can include physical-layer virtualized GPUs and application-layer virtualized GPUs.

[0086] In the solution of the embodiment of the present invention, a plurality of GPUs that are preset to participate in resource scheduling can be formed into a resource pool. The resource pool can include a variety of computing resources. For example, the GPU resources in the resource pool can be GPU container resources, bare metal computing resources, and a plurality of GPUs preset to participate in resource scheduling from third-party computing resources, which are not specifically limited here. The computing resources in the resource pool, including virtualized GPUs and non-virtualized physical GPUs, can be uniformly planned and managed by the GPU resource scheduling method and provide computing power externally.

[0087] Virtualization technology can be divided into physical layer virtualization technology and application layer virtualization technology. A physical layer virtualized GPU is a virtualized GPU obtained through processing using physical layer virtualization technology. Among them, physical layer virtualization technology can virtualize a physical GPU into multiple virtualized GPUs, and these multiple virtualized GPUs can run different algorithms simultaneously, allowing a GPU to run multiple algorithms simultaneously. Physical layer virtualization technology may include virtualization technologies such as Nvdia GRID technology, Nvdia MPS technology, single-card multi-algorithm technology, CUDA hijacking technology, and driver hijacking technology, which are not specifically limited here.

[0088] The underlying computing resources of some physical GPUs may not be virtualized using physical-layer virtualization technology. However, application-layer virtualization technology can be used to control the GPU to run different processing algorithms at different time periods. This allows for scheduling within a preset time period, simulating the ability for a single GPU to run multiple algorithms simultaneously, achieving a macro-level concurrent execution effect. Application-layer virtualization technologies can include time-sharing multiplexing. A virtualized GPU created using application-layer virtualization technology is called an application-layer virtualized GPU.

[0089] The electronic device can allocate a physical layer virtualization GPU, application layer virtualization GPU or physical GPU that can process the task to be processed to the task to be processed based on whether the algorithm that needs to be run when processing the task to be processed is suitable for virtualization technology, and the idle status of the physical GPU, physical layer virtualization GPU and application layer virtualization GPU in the resource pool, thereby achieving resource allocation that is not strongly dependent on the virtualization of the underlying GPU computing resources. Even if the underlying GPU of the computing resources does not support physical layer virtualization, the combination of application layer algorithm scheduling and underlying virtualization technology can be used to enable a single GPU to run multiple algorithms simultaneously, which greatly reduces the limitations of GPU resource scheduling and can flexibly schedule GPU resources.

[0090] As an implementation method of the embodiment of the present invention, Figure 2 As shown, the step of allocating target computing resources from the computing resources to the task to be processed based on the target processing algorithm corresponding to the task to be processed and the idle status of computing resources in the resource pool may include:

[0091] S201, determining whether the target processing algorithm can be run by a virtualized GPU, if yes, executing step S202; if not, executing step S206, i.e., allocating an idle physical GPU from the computing resources to the task to be processed;

[0092] During GPU resource scheduling, there are many different types of pending tasks. Some pending tasks do not require high processing performance, but their number is large, which can be called a "fragmented" scenario. Such pending tasks are suitable for processing by a virtualized GPU. However, some pending tasks have high processing performance requirements. The target processing algorithms of such pending tasks are usually not suitable for physical layer virtualization technology and application layer virtualization technology. They need to be processed by a powerful and independently running physical GPU. Therefore, electronic devices can determine whether the target processing algorithm can be run by a virtualized GPU.

[0093] In one embodiment, if the target processing algorithm cannot be run on a virtualized GPU, the electronic device can send the pending task to an idle physical GPU that has not been virtualized. After receiving the pending task, the physical GPU can process it. If there is no idle physical GPU, the electronic device can temporarily not send the pending task until an idle physical GPU is found, at which time the electronic device can resume GPU resource scheduling.

[0094] S202: Determine whether there is an idle virtualized GPU in the resource pool that meets the scheduling requirements. If yes, execute step S203; if not, execute step S205. That is, temporarily do not issue the pending task;

[0095] If the target processing algorithm can be run by a virtualized GPU, the electronic device may determine, for each virtualized GPU, whether the virtualized GPU has idle resources. Idle resources are unused computing resources within the virtualized GPU. If the virtualized GPU has idle resources, the electronic device may determine whether the idle resources meet the scheduling requirements, specifically, whether the idle resources of the idle virtualized GPU are sufficient to process the pending task. If the idle resources of the idle virtualized GPU are sufficient to process the pending task, the electronic device may proceed to step S203.

[0096] For example, it can be determined by judging whether the computing power of the idle resources can process the pending task. If the idle resources can meet the processing requirements of the pending task, it means that the virtualized GPU meets the conditions for idle resources to process the pending task and can participate in the GPU resource scheduling of the subsequent steps. If there is no idle virtualized GPU with idle resources, or the idle resources of each virtualized GPU do not meet the scheduling requirements of the pending task, the electronic device may temporarily not issue the pending task until there is an idle virtualized GPU with idle resources that meet the scheduling requirements of the pending task, and the electronic device can perform GPU resource scheduling again.

[0097] S203, determining a target virtualized GPU from the idle virtualized GPUs that meet the scheduling requirements;

[0098] S204: Send the pending task to the target virtualized GPU.

[0099] If the electronic device determines that there is an idle virtualized GPU in the resource pool that meets the scheduling requirements, it can determine the target virtualized GPU from the idle virtualized GPUs that meet the scheduling requirements, where the target virtualized GPU can be a physical layer virtualized GPU, that is, the first idle physical layer virtualized GPU. Of course, it can also be an application layer virtualized GPU, that is, the first idle application layer virtualized GPU, which is reasonable.

[0100] After the target virtualized GPU is determined, the electronic device can send the task to be processed to the target virtualized GPU, and then the target virtualized GPU can process the task to be processed accordingly.

[0101] In this embodiment, the electronic device can determine whether the target processing algorithm can be run by a virtualized GPU. If the target processing algorithm can be run by a virtualized GPU, it can determine whether there is an idle virtualized GPU in the resource pool that meets the scheduling requirements. Then, it can determine the target virtualized GPU from the idle virtualized GPUs that meet the scheduling requirements and send the pending task to the target virtualized GPU for processing. In this way, the electronic device can assign pending tasks with high performance requirements to a powerful and independently running physical GPU for processing, and assign fragmented pending tasks with low performance requirements to a virtualized GPU that meets the scheduling requirements for processing. This can reasonably arrange the various pending tasks and avoid wasting computing resources.

[0102] As an implementation of an embodiment of the present invention, the step of determining whether the target processing algorithm can be executed by a virtualized GPU may include:

[0103] Determine whether the target processing algorithm is included in the processing algorithms corresponding to the virtualization technology pre-configured by the virtualized GPU; if included, determine that the target processing algorithm can be executed by the virtualized GPU; if not included, determine that the target processing algorithm cannot be executed by the virtualized GPU.

[0104] The processing algorithms that can be run by the virtualized GPU obtained by virtualization using different virtualization technologies may be different. Therefore, in order to facilitate the determination of whether the target processing algorithm of the task to be processed can be run by the virtualized GPU, the processing algorithms supported by the virtualization technology corresponding to each virtualized GPU can be pre-recorded. In this way, after the electronic device obtains the task to be processed, it can determine whether the pre-recorded processing algorithms include the target processing algorithm corresponding to the task to be processed.

[0105] If the pre-recorded processing algorithms include the target processing algorithm corresponding to the task to be processed, it means that one or more virtualized GPUs can run the target processing algorithm corresponding to the task to be processed, that is, the target processing algorithm can be run by the virtualized GPU. If the pre-recorded processing algorithms do not include the target processing algorithm corresponding to the task to be processed, it means that all virtualized GPUs cannot run the target processing algorithm corresponding to the task to be processed, that is, the target processing algorithm is not run by the virtualized GPU.

[0106] For example, the resource pool includes three virtualized GPUs, among which virtualized GPU1 is obtained by virtualization processing through Nvdia GRID technology, and the processing algorithms corresponding to Nvdia GRID technology are algorithm A, algorithm B and algorithm C; virtualized GPU2 is obtained by virtualization processing through CUDA hijacking technology, and the processing algorithms corresponding to CUDA hijacking technology are algorithm A and algorithm D; virtualized GPU3 is obtained by virtualization processing through time-sharing multiplexing technology, and the processing algorithms corresponding to time-sharing multiplexing technology are algorithm F and algorithm C.

[0107] If the target processing algorithm corresponding to the pending task is Algorithm A, the electronic device can determine that the target processing algorithm can be run by the virtualized GPU because the processing algorithms corresponding to the virtualization technology pre-configured on the virtualized GPU include Algorithm A. If the target processing algorithm corresponding to the pending task is Algorithm E, the electronic device can determine that the target processing algorithm cannot be run by the virtualized GPU because the processing algorithms corresponding to the virtualization technology pre-configured on the virtualized GPU do not include Algorithm E.

[0108] In this embodiment, the electronic device can determine whether the processing algorithms corresponding to the virtualization technology pre-configured on the virtualized GPU include the target processing algorithm. If so, it determines that the target processing algorithm can be run by the virtualized GPU. In this way, based on the pre-recorded processing algorithms corresponding to each virtualized GPU, the electronic device can quickly and accurately determine whether the target processing algorithm can be run by the virtualized GPU, thereby improving the accuracy and efficiency of computing resource scheduling.

[0109] As an implementation method of the embodiment of the present invention, Figure 3 As shown, the step of determining whether there is an idle virtualized GPU in the resource pool that meets the scheduling requirements may include:

[0110] S301: Based on the virtualization ratio of the virtualized GPUs in the resource pool and the occupied computing power of the virtualized GPUs, determine whether there is an idle virtualized GPU in the resource pool; if so, execute step S302; if not, execute step S304, i.e., temporarily not issue the pending task;

[0111] The virtualization ratio of the virtualized GPU in the resource pool is the ratio of the virtualized GPU computing power to the computing power of the resource pool. It can be the first ratio of the virtualized GPU in the computing resources of the resource pool, or it can be the second ratio of the virtualized GPU of the target computing node in the resource pool to the computing resources of the target computing node.

[0112] Based on the virtualization ratio of the virtualized GPU in the resource pool and the total computing power of the resource pool, the electronic device can calculate the total computing power of the virtualized GPU. Furthermore, based on the total computing power of the virtualized GPU and the occupied computing power of the virtualized GPU, the remaining computing power of the virtualized GPU can be calculated, which is the difference between the total computing power of the virtualized GPU and the occupied computing power of the virtualized GPU.

[0113] If the difference is not 0, it means that the remaining computing power of the virtualized GPU is not 0, and the electronic device can determine that there is an idle virtualized GPU in the resource pool. If the difference is 0, it means that the remaining computing power of the virtualized GPU is 0, and the electronic device can determine that there is no idle virtualized GPU in the resource pool.

[0114] S302: Determine whether the remaining computing power of the idle virtualized GPU meets the computing power requirement of the pending task. If so, execute step S303; if not, execute step S304, i.e., temporarily not issue the pending task.

[0115] S303: Determine that the idle virtualized GPU is the idle virtualized GPU that meets the scheduling requirement.

[0116] If there is an idle virtualized GPU in the resource pool, the electronic device can determine whether the remaining computing power of the idle virtualized GPU meets the computing power requirements of the task to be processed. If so, it means that the idle virtualized GPU can process the task to be processed, and the electronic device can determine that the idle virtualized GPU is an idle virtualized GPU that meets the scheduling requirements.

[0117] If there is no idle virtualized GPU, or the remaining computing power of the idle virtualized GPU cannot meet the computing power requirements of the task to be processed, the electronic device may temporarily not send the task to be processed, and wait until there is an idle virtualized GPU and the remaining computing power of the idle virtualized GPU meets the computing power requirements of the task to be processed, and then send the task to be processed.

[0118] In this embodiment, the electronic device can determine whether there are idle virtualized GPUs in the resource pool based on the virtualization ratio of the virtualized GPUs in the resource pool and the occupied computing power of the virtualized GPUs. If there are idle virtualized GPUs, the electronic device can further determine whether the remaining computing power of the idle virtualized GPUs meets the computing power requirements of the task to be processed. If so, the idle virtualized GPU can be determined as an idle virtualized GPU that meets the scheduling requirements. In this way, the electronic device can accurately determine the idle virtualized GPU that meets the scheduling requirements.

[0119] As an implementation manner of an embodiment of the present invention, the step of determining the target virtualized GPU from the idle virtualized GPUs that meet the scheduling requirements may include:

[0120] Determine whether there is a first virtualized GPU that supports the target physical layer virtualization technology among the idle virtualized GPUs that meet the scheduling requirements; if there are multiple first virtualized GPUs, determine the first idle physical layer virtualization GPU from the first virtualized GPUs based on the priority of the physical layer virtualization technology corresponding to each first virtualized GPU.

[0121] Because processing algorithms in application-layer virtualization technology run in time zones, there are certain requirements for the coordination of multiple processing algorithms. In one embodiment, if two processing algorithms have a dependency relationship, it is necessary to determine whether the two algorithms can be coordinated. If it is determined that the two algorithms can be coordinated, the execution order of the two algorithms is set.

[0122] For example, if processing algorithms A and B require the same data simultaneously, the virtualized GPU created using application-layer virtualization technology may not be able to run both algorithms simultaneously. Consequently, algorithms A and B cannot be combined. If both algorithms A and B are run on the same virtualized GPU, the algorithms will die due to algorithm starvation. For another example, suppose that processing algorithms C and D can be combined. The dependency between them is that the analysis results of processing algorithm C are required during the execution of processing algorithm D. Therefore, processing algorithm C must be executed before processing algorithm D. Since the virtualized GPU created using application-layer virtualization technology calls different processing algorithms in a time-sharing manner, the execution order of processing algorithms C and D can be set to: execute processing algorithm C first, then execute processing algorithm D. This prevents idle processing algorithms from wasting computing resources.

[0123] The multiple virtualized GPUs virtualized using physical layer virtualization technology can be considered to execute processing algorithms concurrently or in parallel, with fewer restrictions on the processing algorithms. Virtualized GPUs virtualized using physical layer virtualization technology have high computational efficiency, good isolation between processing algorithms, and more flexible combinations of processing algorithms. Therefore, when scheduling GPU resources, virtualized GPUs that support physical layer virtualization technology can be prioritized. Therefore, electronic devices can determine whether a first virtualized GPU that supports the target physical layer virtualization technology is among the idle virtualized GPUs that meet the scheduling requirements.

[0124] Among them, the target physical layer virtualization technology is the physical layer virtualization technology applicable to the target processing algorithm corresponding to the task to be processed, that is, after the physical GPU is virtualized through the target physical layer virtualization technology to obtain a virtualized GPU, the target processing algorithm can run on the virtualized GPU.

[0125] Since it has been determined that the idle resources of the idle virtualized GPUs can meet the scheduling requirements of the pending tasks, the electronic device can determine whether there is a first virtualized GPU that supports the target physical layer virtualization technology among the idle virtualized GPUs.

[0126] As an implementation method, the electronic device can determine whether the pre-set physical layer virtualization technology label corresponding to the idle virtualized GPU matches the label of the target physical layer virtualization technology. If so, it is determined that there is a first virtualized GPU that supports the target physical layer virtualization technology; if not, it is determined that there is no first virtualized GPU that supports the target physical layer virtualization technology.

[0127] Because a GPU can be virtualized using a variety of physical layer virtualization technologies, a physical layer virtualization technology tag of the virtualized GPU can be pre-set to distinguish different physical layer virtualization technologies that can be used to virtualize the GPU. The physical layer virtualization technology tag is used to identify the physical layer virtualization technology supported by the virtualized GPU.

[0128] For example, if a physical GPU is virtualized using target physical layer virtualization technology A to obtain multiple virtualized GPUs, the electronic device can set the physical layer virtualization technology label of these multiple virtualized GPUs to "A." Similarly, if the physical GPU is virtualized using target physical layer virtualization technology B to obtain multiple virtualized GPUs, the electronic device can set the physical layer virtualization technology labels of these multiple virtualized GPUs to "A" and "B." A virtualized GPU can have a single physical layer virtualization technology label or multiple different physical layer virtualization technology labels.

[0129] If the physical layer virtualization technology tag corresponding to the idle virtualized GPU matches the tag of the target physical layer virtualization technology, the electronic device can determine that the idle virtualized GPU supports the target physical layer virtualization technology, and further determine that there is a first virtualized GPU that supports the target physical layer virtualization technology. For example, if the physical layer virtualization technology tags corresponding to the idle virtualized GPU are "A," "B," and "C," and the tag of the target physical layer virtualization technology is "B," it can be determined that the physical layer virtualization technology tag corresponding to the idle virtualized GPU matches the tag of the target physical layer virtualization technology, and the electronic device can determine that there is a first virtualized GPU that supports the target physical layer virtualization technology.

[0130] If the physical layer virtualization technology tag corresponding to the idle virtualized GPU does not match the tag of the target physical layer virtualization technology, the electronic device can determine that there is no first virtualized GPU that supports the target physical layer virtualization technology. For example, if the physical layer virtualization technology tags corresponding to the idle virtualized GPU are "A," "B," and "C," and the tag of the target physical layer virtualization technology is "D," it can be determined that the physical layer virtualization technology tag corresponding to the idle virtualized GPU does not match the tag of the target physical layer virtualization technology. The electronic device can determine that the idle virtualized GPU does not support the target physical layer virtualization technology and is therefore not the first virtualized GPU.

[0131] If there is only one first virtualized GPU, the electronic device may determine the first virtualized GPU as the first idle physical layer virtualized GPU. If there are multiple first virtualized GPUs, to better process the pending task, the electronic device may determine the first idle physical layer virtualized GPU from among the first virtualized GPUs based on the priority of the physical layer virtualization technology corresponding to each first virtualized GPU.

[0132] Because GPUs can be virtualized using a variety of physical layer virtualization technologies, the processing performance of the virtualized GPUs obtained by virtualizing the same GPU using different physical layer virtualization technologies may vary. To distinguish the differences in processing performance of virtualized GPUs obtained using different physical layer virtualization technologies, the priority of the physical layer virtualization technologies corresponding to the virtualized GPUs can be pre-set.

[0133] Among them, the priority of the physical layer virtualization technology can be determined based on the processing performance of the simulated virtualized GPU corresponding to the first virtualized GPU. The simulated virtualized GPU is obtained by virtualizing the GPU corresponding to the first virtualized GPU through multiple physical layer virtualization technologies.

[0134] For example, GPU1 can be virtualized through CUDA hijacking technology or driver hijacking technology. The simulated virtualized GPUs obtained by virtualization through CUDA hijacking technology are GPU11 and GPU12; the simulated virtualized GPUs obtained by virtualizing GPU1 through driver hijacking technology are GPU21 and GPU22. The difference between the processing performance of one of GPU11 and GPU12 and one of GPU21 and GPU22 is compared. If the processing performance of GPU11 is better than the processing performance of GPU21, it can be determined that the priority of the physical layer virtualization technology corresponding to the virtualized GPU11 is CUDA hijacking technology higher than driver hijacking technology.

[0135] After determining the priority of the physical layer virtualization technology corresponding to each first virtualized GPU, the electronic device can determine the first virtualized GPU with the highest corresponding priority from multiple first virtualized GPUs that support the target physical layer virtualization technology as the first idle physical layer virtualization GPU, and the first idle physical layer virtualization GPU is the target virtualized GPU.

[0136] For example, there are three first virtualized GPUs that support the target physical layer virtualization technology, and the target physical layer virtualization technology is CUDA hijacking technology. The three first virtualized GPUs are virtualized GPU31, virtualized GPU41, and virtualized GPU51. Among them, the priorities of the physical layer virtualization technologies corresponding to virtualized GPU31 are, from high to low, CUDA hijacking technology, driver hijacking technology, and one-card multi-algorithm technology; the priorities of the physical layer virtualization technologies corresponding to virtualized GPU41 are, from high to low, driver hijacking technology, CUDA hijacking technology, and one-card multi-algorithm technology; the priorities of the physical layer virtualization technologies corresponding to virtualized GPU51 are, from high to low, one-card multi-algorithm technology, driver hijacking technology, and CUDA hijacking technology. Since the physical layer virtualization technology with the highest priority among the priorities of the physical layer virtualization technologies corresponding to virtualized GPU31 is the target physical layer virtualization technology, namely, CUDA hijacking technology, the electronic device can determine that virtualized GPU31 is the first idle physical layer virtualization GPU.

[0137] In this embodiment, when there are multiple first virtualized GPUs that meet the scheduling requirements of the tasks to be processed and support the target physical layer virtualization technology, the electronic device can determine a first idle physical layer virtualization GPU from the first virtualized GPUs based on the priority of the physical layer virtualization technology corresponding to each first virtualized GPU. Through the above scheme, the electronic device can determine the first idle physical layer virtualization GPU based on the priority of the physical layer virtualization technology corresponding to the first virtualized GPU, thereby enabling the first idle physical layer virtualization GPU with the target physical layer virtualization technology to process the tasks to be processed preferentially through the corresponding high-priority physical layer virtualization technology, thereby improving the processing efficiency of the tasks to be processed and rationally scheduling GPU computing resources.

[0138] As an implementation method of the embodiment of the present invention, Figure 4 As shown, the priority setting method of the above-mentioned physical layer virtualization technology may include:

[0139] S401 : For each first virtualized GPU, virtualize the GPU corresponding to the first virtualized GPU using multiple physical layer virtualization technologies to obtain a corresponding simulated virtualized GPU.

[0140] When a GPU supports multiple physical layer virtualization technologies, the processing performance of virtualized GPUs obtained by virtualizing the GPU using different physical layer virtualization technologies may vary significantly. To determine the difference in processing performance between virtualized GPUs obtained using different physical layer virtualization technologies, the electronic device may virtualize the GPU corresponding to each first virtualized GPU using multiple physical layer virtualization technologies to obtain corresponding simulated virtualized GPUs.

[0141] S402: Determine the priority of the physical layer virtualization technology corresponding to the simulated virtualized GPU based on the processing performance of the simulated virtualized GPU.

[0142] The electronic device can compare the differences in processing performance between the simulated virtualized GPUs corresponding to the same virtualized GPU, thereby determining the priority of the physical layer virtualization technology corresponding to each simulated virtualized GPU. As an implementation method, the electronic device can sort the physical layer virtualization technologies corresponding to the simulated virtualized GPU in order from high to low according to processing performance, thereby determining the priority of the physical layer virtualization technology corresponding to the simulated virtualized GPU.

[0143] For example, GPU2 supports CUDA hijacking technology and one-card multi-algorithm technology. By virtualizing GPU2 through CUDA hijacking technology, simulated virtualized GPU21 and GPU22 can be obtained; by virtualizing GPU2 through one-card multi-algorithm technology, simulated virtualized GPU23 and GPU24 can be obtained. The processing performance of simulated virtualized GPU21 can be compared with that of simulated virtualized GPU23. If the processing performance of simulated virtualized GPU21 is better than that of simulated virtualized GPU23, then the physical layer virtualization technology priority of the first virtualized GPU obtained by processing GPU2 is CUDA hijacking technology over one-card multi-algorithm technology.

[0144] In this embodiment, the electronic device can virtualize the GPU corresponding to the first virtualized GPU using multiple physical layer virtualization technologies to obtain a corresponding simulated virtualized GPU, and then determine the priority of the physical layer virtualization technology corresponding to the simulated virtualized GPU based on the processing performance of the simulated virtualized GPU. In this way, the electronic device can accurately determine the priority of the physical layer virtualization technology corresponding to each first virtualized GPU, so that the first idle physical layer virtualization GPU with higher processing performance can subsequently process the pending task, thereby improving the efficiency and effectiveness of task processing.

[0145] As an implementation method of the embodiment of the present invention, Figure 5As shown, the step of determining the target virtualized GPU from the idle VPUs that meet the scheduling requirements may further include:

[0146] S501 , determining whether there is a second virtualized GPU that supports the target application layer virtualization technology among the idle virtualized GPUs that meet the scheduling requirements. If yes, executing step S502 or step S503 .

[0147] If the first virtualized GPU that supports the target physical layer virtualization technology does not exist among the idle virtualized GPUs that meet the scheduling requirements, this indicates that although the idle virtualized GPU has idle resources that meet the scheduling requirements of the pending task, the pending task cannot be run on the idle virtualized GPU. The electronic device can then determine whether the second virtualized GPU that supports the target application layer virtualization technology exists among the idle virtualized GPUs that meet the scheduling requirements.

[0148] The second virtualized GPU is a virtualized GPU virtualized using an application-layer virtualization technology, and the target application-layer virtualization technology is the application-layer virtualization technology applicable to the target processing algorithm. If it has been determined that the idle virtualized GPU has idle resources and that the idle resources meet the scheduling requirements of the pending task, then if there is a second virtualized GPU that supports the target application-layer virtualization technology, the second virtualized GPU is likely capable of processing the pending task, and the electronic device may execute step S502 or step S503.

[0149] S502: For a computing node having the second virtualized GPU, determine whether the second virtualized GPU corresponding to the computing node is running other processing algorithms that are pre-bound and associated with the target processing algorithm; if so, determine that the second virtualized GPU corresponding to the computing node is the first idle application layer virtualized GPU; if not, determine whether the second virtualized GPU corresponding to the next computing node is running other processing algorithms that are pre-bound and associated with the target processing algorithm; until it is determined that all second virtualized GPUs are no longer running other processing algorithms that are pre-bound and associated with the target processing algorithm, determine the second virtualized GPU that supports the target processing algorithm as the first idle application layer virtualized GPU;

[0150] The second virtualized GPU is a virtualized GPU virtualized using target application layer virtualization technology. The target processing algorithm and other processing algorithms associated with the target processing algorithm can be bound to the second virtualized GPU. In this way, the second virtualized GPU can run the target processing algorithm when processing the pending task.

[0151] Because the second virtualized GPU runs different processing algorithms at different time periods, if each pending task is processed by a separate second virtualized GPU, a large amount of the second virtualized GPU will be occupied, resulting in the inability to process tasks corresponding to other processing algorithms not bound to the target processing algorithm, which will seriously reduce task processing efficiency. If the second virtualized GPU only runs the target processing algorithm, computing power will be wasted. Therefore, the electronic device can determine whether the second virtualized GPU is running other processing algorithms pre-bound and associated with the target processing algorithm, and based on the determination result, determine the second application layer virtualized GPU to be used to process the pending task, thereby fully utilizing the computing power of the second virtualized GPU.

[0152] In one embodiment, the first idle application layer virtualized GPU can be determined based on the computing node. Specifically, the electronic device can determine, for a computing node where a second virtualized GPU exists, whether the second virtualized GPU corresponding to the computing node is running other processing algorithms that are pre-bound and associated with the target processing algorithm.

[0153] If the second virtualized GPU corresponding to the computing node is running other processing algorithms pre-bound and associated with the target processing algorithm, the electronic device can determine that the second virtualized GPU corresponding to the computing node is a first idle application layer virtualized GPU for processing tasks to be processed.

[0154] If the second virtualized GPU corresponding to the computing node is not running other processing algorithms that are pre-bound and associated with the target processing algorithm, that is, the computing node does not have a second virtualized GPU that is running other processing algorithms that are pre-bound and associated with the target processing algorithm, in order to improve task processing efficiency, the electronic device can determine whether the second virtualized GPU corresponding to the next computing node is running other processing algorithms that are pre-bound and associated with the target processing algorithm. If so, the second virtualized GPU corresponding to the computing node is set as the first idle application layer virtualized GPU.

[0155] If not, continue to determine whether the second virtualized GPU corresponding to the next computing node is running other processing algorithms that are pre-bound and associated with the target processing algorithm. If it is determined that all second virtualized GPUs are not running other processing algorithms that are pre-bound and associated with the target processing algorithm, the electronic device can determine the second virtualized GPU that supports the target processing algorithm as the first idle application layer virtualized GPU for processing the tasks to be processed.

[0156] If there are multiple second virtualized GPUs that support the target processing algorithm, the electronic device can randomly select one of them as the first idle application layer virtualized GPU. Alternatively, the electronic device can select one of the multiple second virtualized GPUs that support the target processing algorithm as the first idle application layer virtualized GPU according to a preset method.

[0157] For example, according to the remaining computing power of the second virtualized GPU or the remaining computing power of the computing node where it is located, a second virtualized GPU that can efficiently process the tasks to be processed can be selected as the first idle application layer virtualized GPU to improve the processing efficiency of the tasks to be processed.

[0158] S503: If there are multiple second virtualized GPUs in the resource pool, for each second virtualized GPU, determine whether the second virtualized GPU is running other processing algorithms that are pre-bound and associated with the target processing algorithm; if so, determine that the second virtualized GPU is the first idle application layer virtualized GPU; if not, determine that the second virtualized GPU that supports the target processing algorithm is the first idle application layer virtualized GPU.

[0159] The electronic device can also determine the first idle application layer virtualized GPU based on the second virtualized GPU. Specifically, for each second virtualized GPU, the electronic device can determine whether the second virtualized GPU is running other processing algorithms that are pre-bound and associated with the target processing algorithm. If so, it can be determined that the second virtualized GPU is the first idle application layer virtualized GPU.

[0160] If none of the second virtualized GPUs are running any other pre-bound processing algorithms associated with the target processing algorithm, the electronic device may determine that the second virtualized GPU supporting the target processing algorithm is the first idle application-layer virtualized GPU. The specific implementation for determining that the second virtualized GPU supporting the target processing algorithm is the first idle application-layer virtualized GPU is the same as in step S502 and is not further described here.

[0161] In this embodiment, the electronic device can determine whether there is a second virtualized GPU running other pre-bound processing algorithms associated with the target processing algorithm based on the computing node or the second virtualized GPU. If so, the electronic device can determine the second virtualized GPU as the first idle application layer virtualized GPU to process the above-mentioned pending tasks. On the basis of conventional time-sharing multiplexing virtualization technology, the binding of multiple processing algorithms is added, which can realize the functions of processing algorithm execution order and time slice scheduling. Under the premise of solving the algorithm thirst, the GPU computing resources used are reasonably configured according to the usage scenario of the processing algorithm. When the physical layer GPU virtualization does not support the target GPU virtualization technology, the purpose of "one card multiple algorithms" is achieved at the GPU application layer to avoid waste of computing power.

[0162] As an implementation method of the embodiment of the present invention, Figure 6 As shown, the binding method of the above processing algorithm may include:

[0163] S601 : For each second virtualized GPU, determine a processing algorithm required for processing a required image task.

[0164] In the process of processing image tasks, the virtualized GPU usually needs to run multiple processing algorithms, and these multiple processing algorithms are often related. The related processing algorithms can be bound to the same second virtualized GPU. For each second virtualized GPU, the electronic device can determine the processing algorithm required to process the image task to be processed. For example, if the image task is to extract facial features from an image, then in the process of extracting facial features, the algorithms that need to be run may include multiple processing algorithms such as face alignment algorithm, face detection algorithm, image boundary extraction algorithm, and facial feature extraction, and the above processing algorithms may be interrelated. The above-mentioned task to be processed may be one of the image tasks.

[0165] S602: Bind the processing algorithm to the second virtualized GPU according to the processing order of the image tasks.

[0166] After determining the processing algorithms required to process the image task to be processed, the electronic device can bind the processing algorithms to the second virtualized GPU in the order in which the image tasks are processed. The second virtualized GPU can then run the processing algorithms based on this order. For example, if the image task is to extract a face image from a picture, the electronic device can bind the corresponding face alignment algorithm, face detection algorithm, image boundary extraction algorithm, and face feature extraction algorithm to the second virtualized GPU in the order in which the image tasks are processed: face alignment, face detection, image boundary extraction, and face feature extraction.

[0167] S603 : Allocate the execution time corresponding to each processing algorithm based on the time ratio required by the second virtualized GPU to execute each processing algorithm.

[0168] Because different processing algorithms require different computing power, the second virtualized GPU may require different amounts of time to execute each processing algorithm. The electronic device may allocate a corresponding execution time for each processing algorithm based on the ratio of the time required by the second virtualized GPU to execute each processing algorithm. The execution time is the duration that the second virtualized GPU executes the corresponding processing algorithm. The second virtualized GPU may execute each processing algorithm according to the allocated execution time based on the order in which the processing algorithms are bound.

[0169] Compared to sending the task to be processed to different virtualized GPUs and loading different processing algorithms, which takes a lot of time, it only takes about 3-10 seconds for the electronic device to control the second virtualized GPU to switch between different pre-bound processing algorithms. For example, taking the image task of extracting a face image from a picture as an example, the time ratio of the second virtualized GPU running the face alignment algorithm, face detection algorithm, image boundary extraction algorithm, and face feature extraction algorithm is 1:2:1:2. Therefore, the preset time period can be 6 minutes, of which the face alignment algorithm runs for 1 minute; the face detection algorithm runs for 2 minutes; the image boundary extraction algorithm runs for 1 minute; and the face feature extraction algorithm runs for 2 minutes.

[0170] If the task to be processed is an image boundary extraction task, and the corresponding target processing algorithm is an image boundary extraction algorithm, when the electronic device sends the image boundary extraction task to the second virtualized GPU, if the second virtualized GPU is starting to run the face alignment algorithm, then after the second virtualized GPU runs the face alignment algorithm for 1 minute, the electronic device can switch the processing algorithm of the second virtualized GPU to a face detection algorithm, and after running the face detection algorithm for 2 minutes, the electronic device can switch the processing algorithm of the second virtualized GPU to an image boundary extraction algorithm, so that the second virtualized GPU can start running the image boundary extraction algorithm, and the running time is 1 minute.

[0171] In this embodiment, the electronic device can determine, for each second virtualized GPU, the processing algorithm required to process the required image task, bind the processing algorithms to the second virtualized GPU in the order in which the image tasks are processed, and allocate the corresponding runtime for each processing algorithm based on the ratio of time required for the second virtualized GPU to run each processing algorithm. This allows the second virtualized GPU to reasonably set the runtime for multiple processing algorithms based on their application scenarios, avoiding the waste of computing power caused by improperly set runtimes for processing algorithms and improving the efficiency of image task processing.

[0172] As an implementation of an embodiment of the present invention, the above method may further include:

[0173] If the first virtualized GPU and the second virtualized GPU do not exist, the to-be-processed task is sent to an idle physical GPU, so that the physical GPU processes the to-be-processed task.

[0174] During GPU resource scheduling, if neither the first virtualized GPU nor the second virtualized GPU exists, it indicates that all virtualized GPU resources capable of running the target processing algorithm corresponding to the pending task are occupied. In this case, the pending task must be processed using a non-virtualized physical GPU. The electronic device can send the pending task to an idle physical GPU, which has not been virtualized and has higher processing performance. Upon receiving the pending task, the idle physical GPU can process it.

[0175] In this embodiment, if neither the first virtualized GPU nor the second virtualized GPU exists, the electronic device can send the pending task to an idle physical GPU. Upon receiving the pending task, the idle physical GPU can then process it. This ensures that even if the virtualized GPU is currently unable to process the pending task, the idle physical GPU can still handle it, ensuring timely processing of the pending task and improving system resource utilization.

[0176] As an implementation of an embodiment of the present invention, before the step of obtaining the tasks to be processed, the method may further include:

[0177] Virtualization is performed on the GPU in the resource pool to obtain the computing resources.

[0178] The resource pool includes a plurality of preset GPUs that participate in resource scheduling, and the proportion of virtualized GPUs in the computing resources of the resource pool satisfies a first ratio. Furthermore, the proportion of virtualized GPUs in the computing resources of the target computing node in the resource pool satisfies a second ratio.

[0179] Electronic devices can virtualize GPUs in a resource pool based on a pre-set virtualization ratio. Virtualization can be prioritized through physical layer virtualization technology to improve the computing efficiency of the virtualized GPUs, enhance the isolation between processing algorithms, and more flexibly combine multiple processing algorithms.

[0180] In one embodiment, the electronic device can virtualize the GPU in the resource pool to obtain computing resources based on a pre-set overall virtualization ratio of the resource pool computing resources, i.e., a first ratio. The first ratio can be set based on actual conditions such as whether the GPU in the resource pool supports virtualization processing and the computing power of the virtualized GPU required for the task to be processed. For example, it can be 20%, 50%, 80%, etc., which is not specifically limited here. If the pre-set first ratio is 80%, the electronic device can virtualize the GPU in the resource pool through physical layer virtualization technology or application layer virtualization technology until the computing power of the obtained virtualized GPU reaches 80% of the total computing power of the resource pool.

[0181] In another embodiment, one or more GPUs in a resource pool can serve as a computing node, providing computing power to the outside world as a whole. Therefore, the electronic device can perform GPU virtualization on the computing nodes in the resource pool according to a preset second ratio, i.e., the ratio of the virtualized GPU in the computing resources of the target computing node in the resource pool to the computing resources of the target computing node. The ratio satisfies the second ratio.

[0182] When the compute node has a single GPU, the GPU can be virtualized, either through physical-layer virtualization or application-layer virtualization technology, resulting in a virtualized GPU. Alternatively, the GPU can be virtualized and provided computing power as an independent GPU. When the compute node has multiple GPUs, a second virtualization ratio can be set for the compute node. For example, if the second virtualization ratio for the compute node is 20%, and the compute node has 10 GPUs with identical hardware parameters and processing performance, two of them can be virtualized, while the remaining eight remain unvirtualized, with each GPU corresponding to a processing algorithm.

[0183] Taking two GPUs undergoing virtualization as an example, these two GPUs can be virtualized using either physical layer virtualization or application layer virtualization. To achieve higher processing performance, physical layer virtualization is prioritized if the GPU supports it. If the GPU does not support it, application layer virtualization is used to virtualize the GPU.

[0184] In this embodiment, the electronic device can configure and perform virtualization processing on the proportion of virtualized resources in the entire resource pool, or can configure and perform virtualization processing on the proportion of virtualized resources of a single computing node. Fragmented scenarios with low performance requirements can be configured on reasonable computing resources to avoid performance waste, thereby more reasonably managing VGPU computing resources and enhancing the performance and scalability of the resource pool.

[0185] As an implementation manner of an embodiment of the present invention, the step of performing virtualization processing on the GPU in the resource pool to obtain the computing resources may include:

[0186] Sorting the multiple GPUs in the resource pool according to processing performance to obtain a sorting result; based on the sorting result, virtualizing the GPUs in order of processing performance from low to high until the proportion of the computing power of the obtained virtualized GPUs in the resource pool reaches the first proportion or the second proportion.

[0187] The resource pool includes multiple GPUs that are pre-configured to participate in resource scheduling. The GPUs' hardware parameters and processing performance may vary. Compared to high-performance GPUs, GPUs with lower processing power have lower computing power and are unable to process tasks requiring higher processing power. GPUs with lower processing power are more suitable for processing fragmented tasks requiring less processing power, so GPUs with lower processing power are prioritized for virtualization. The electronic device can sort the multiple GPUs in the resource pool by processing performance to obtain a ranking result.

[0188] Based on the sorting results, the electronic device can virtualize the GPUs in order of processing performance from low to high, with the lower the GPU's processing performance, the earlier it is virtualized. Because GPUs with lower processing performance are prioritized for virtualization, until the proportion of the computing power of the resulting virtualized GPUs in the resource pool reaches the first or second proportion, some GPUs with higher processing performance in the resource pool are retained and not virtualized. Because GPUs with higher processing performance are not virtualized, they can be used to process pending tasks that require higher processing performance.

[0189] In this embodiment, the electronic device can sort multiple GPUs in a resource pool by processing performance to obtain a sorting result. Based on the sorting result, the electronic device virtualizes the GPUs in ascending order of processing performance until the computing power of the resulting virtualized GPUs in the resource pool reaches a first ratio or a second ratio. In this way, the GPU with lower processing performance, which is suitable for fragmented tasks, is virtualized to obtain a virtualized GPU; the GPU with higher processing performance is not virtualized and can be used to process tasks requiring higher processing performance, thus fully utilizing the GPU resources with different processing performance.

[0190] The structure diagram of the GPU scheduling system using the embodiment of the present invention can be as follows: Figure 7 As shown, the GPU scheduling system may include the following components:

[0191] Algorithm orchestration service 701: Algorithm orchestration service 701 can orchestrate information of pending tasks and send the orchestrated pending tasks to algorithm scheduling service 702.

[0192] Algorithm scheduling service 702 : The algorithm scheduling service 702 can execute the above-mentioned GPU resource scheduling method, determine which virtualized GPU or physical GPU will process the pending task, and send the pending task to the resource pool service 703 .

[0193] Resource pool service 703: The resource pool service 703 includes a variety of GPU computing resources, such as bare metal computing resources and container computing resources, among which the GPU in the resource pool service 703 can be a virtualized GPU obtained by virtualization processing through GPU virtualization technology, as well as a physical GPU that has not been virtualized. GPU virtualization technology may include Nvdia GRID technology, Nvdia MPS technology, single-card multi-algorithm technology, CUDA hijacking technology and driver hijacking technology, etc.; after the resource pool service 703 receives the pending tasks, it can forward the pending tasks to the virtualized GPU or physical GPU determined by the algorithm scheduling service 702.

[0194] Smart device: The smart device is a control device for GPU computing resources. When a virtualized GPU or a physical GPU receives a task to be processed, it can divide the computing power according to the corresponding virtualization technology 704 such as one-card multi-algorithm, CUDA hijacking, etc., and control the virtualized GPU or the physical GPU to start processing the task to be processed, thereby enabling the same GPU to run different processing algorithms.

[0195] As an implementation method of the embodiment of the present invention, the following Figure 8 A GPU resource scheduling system that applies the GPU resource scheduling method provided by an embodiment of the present invention is introduced.

[0196] The algorithm orchestration service can orchestrate information about pending tasks and send the orchestrated pending tasks to the algorithm scheduling service. Among them, algorithm orchestration service 1 can be used to orchestrate pending tasks with higher performance requirements, while algorithm orchestration service 2 can be used to orchestrate pending tasks with lower performance requirements.

[0197] The algorithm scheduling service can execute the GPU resource scheduling method to determine whether the received pending tasks are processed by the virtualized GPU or the physical GPU; if they are processed by the virtualized GPU, the algorithm scheduling service can determine whether they are processed by the virtualized GPU that supports physical layer virtualization technology or by the virtualized GPU that supports application layer virtualization technology, and send the pending tasks to the resource pool so that the corresponding GPU computing resources can process the pending tasks.

[0198] The resource pool includes a variety of GPU computing resources, which can be physical GPUs, virtualized GPUs that support physical layer virtualization technology, and virtualized GPUs that support application layer virtualization technology.

[0199] In GPU computing resources, VGPUs are virtualized GPUs that support physical layer virtualization technology. A physical GPU can be virtualized using physical layer virtualization technology to generate multiple VGPUs, which can run different processing algorithms simultaneously. For example, if physical layer virtualization generates five VGPUs, the first VGPU can run algorithm A, the second VGPU can run algorithm B, the third VGPU can run algorithm C, the fourth VGPU can run algorithm D, and the fifth VGPU can run algorithm E at the same time.

[0200] Virtualized processing is performed through application-layer virtualization technology, resulting in a virtualized GPU. Multiple related processing algorithms can be bound to the virtualized GPU. For example, the F algorithm, the G algorithm, the H algorithm, and the I algorithm are bound together. The execution order is F algorithm first, followed by G algorithm, then H algorithm, and finally I algorithm. The execution time of these multiple processing algorithms can be pre-allocated based on the ratio of time required for the virtualized GPU to execute each processing algorithm.

[0201] The flowchart of the GPU virtualization process based on the above embodiment can be as follows: Figure 9 Shown, including:

[0202] S901: Start GPU virtualization configuration.

[0203] S902: Determine a processing algorithm that supports GPU virtualization.

[0204] Specifically, the electronic device can obtain all algorithm information in the scheduling system and select and configure processing algorithms that support GPU virtualization. These selected and configured processing algorithms are the processing algorithms that can be run by the virtualized GPU and can subsequently be used to determine whether the target processing algorithm corresponding to the task to be processed can be run by the virtualized GPU.

[0205] S903: Determine whether to configure the virtualization ratio of the resource pool.

[0206] The electronic device may determine whether the virtualization ratio of the resource pool needs to be configured. If yes, step S904 is executed; if no, step S905 is executed.

[0207] S904: Configure the virtualization ratio of the resource pool.

[0208] If it is necessary to configure the virtualization ratio of the resource pool, the electronic device can configure the virtualization ratio of the resource pool. For example, it can be the above-mentioned first ratio, which can be used to subsequently determine whether there is an idle virtualized GPU in the resource pool that meets the scheduling requirements. The specific implementation methods have been described in the above-mentioned embodiments and will not be repeated here.

[0209] In one embodiment, the electronic device may sort multiple GPUs in the resource pool according to processing performance to obtain a sorting result, and then, based on the sorting result, virtualize the GPUs in ascending order of processing performance until the proportion of virtualized GPUs in the resource pool reaches a first proportion. After configuring the virtualization ratio of the resource pool, the electronic device may execute step S907.

[0210] S905: Select a computing node that requires virtualization configuration.

[0211] If the virtualization ratio of the resource pool is not configured, the electronic device can select a computing node that needs to be virtualized from the resource pool.

[0212] S906: Configure the virtualization ratio of the selected computing node.

[0213] The virtualization ratio may be a second ratio of the virtualized GPU of the compute node in the resource pool to the computing resources of the compute node, and may be used to subsequently determine whether there are idle virtualized GPUs in the resource pool that meet the scheduling requirements. The specific implementation methods have been described in the above embodiments and will not be repeated here. After configuring the virtualization ratio of the selected compute node, the electronic device may execute step S907.

[0214] S907: Determine whether the GPU supports physical layer virtualization.

[0215] For each GPU, the electronic device may determine whether the GPU supports physical layer virtualization. If yes, execute step S908; if no, execute step S911.

[0216] S908: Determine the physical layer virtualization technology label of the virtualized GPU.

[0217] If the GPU supports physical layer virtualization, the electronic device may determine the physical layer virtualization technology tag of the virtualized GPU. For example, physical layer virtualization technologies may include NVIDIA GRID technology, NVIDIA MPS technology, single-card multi-algorithm technology, CUDA hijacking technology, and driver hijacking technology. The corresponding physical layer virtualization technology tags may be A, B, C, D, and E, respectively. After determining the physical layer virtualization technology tag of the virtualized GPU, the electronic device may proceed to step S909.

[0218] S909: Determine whether the virtualized GPU supports multiple physical layer virtualization technologies.

[0219] For each virtualized GPU, the electronic device may determine whether the virtualized GPU supports multiple physical layer virtualization technologies. If yes, execute step S910; if not, execute step S915.

[0220] S910: Configure the priorities of multiple physical layer virtualization technologies.

[0221] If the virtualized GPU supports multiple physical layer virtualization technologies, the electronic device can configure the priorities of the multiple physical layer virtualization technologies. The specific implementation method has been described in detail in the above embodiment and will not be repeated here; then the electronic device can execute step S915.

[0222] S911: Determine whether the GPU supports application layer virtualization.

[0223] If the GPU does not support physical layer virtualization, the electronic device may determine whether the GPU supports application layer virtualization for the GPU that does not support physical layer virtualization. If so, execute step S912; if not, execute step S915.

[0224] S912: Determine multiple associated processing algorithms.

[0225] If the GPU supports application layer virtualization, the electronic device may determine a plurality of associated processing algorithms required for processing the image task to be processed, and then execute step S913.

[0226] S913: Associating the processing algorithm with the GPU according to the processing order.

[0227] The electronic device may bind the associated multiple processing algorithms to the GPU in a processing order, so that when subsequently processing pending tasks, the application-layer virtualized GPU used to process the pending tasks can be determined based on the processing algorithms bound to each application-layer virtualized GPU. The electronic device may then proceed to step S914.

[0228] S914: Allocate the running time of each processing algorithm.

[0229] The electronic device may allocate the corresponding running time of each processing algorithm based on the time ratio required to run each processing algorithm, and then execute step S915.

[0230] S915: GPU virtualization configuration is complete.

[0231] After configuration is complete, computing resources include physical GPUs and virtualized GPUs. Virtualized GPUs include physical-layer virtualized GPUs and application-layer virtualized GPUs. In the subsequent scheduling process, physical-layer virtualized GPUs, application-layer virtualized GPUs, or physical GPUs can be selected to process pending tasks based on actual conditions.

[0232] In this embodiment, the electronic device can determine a processing algorithm suitable for GPU virtualization, and perform virtualization processing on the GPU of the resource pool or the computing node in the resource pool based on the virtualization ratio. During the physical layer virtualization processing, the physical layer virtualization technology label of the virtualized GPU obtained by the physical layer virtualization processing can be determined, and the priority of multiple physical layer virtualization technologies can be configured. When using application layer virtualization technology, the electronic device can determine the associated multiple processing algorithms required for processing the target task to be processed, bind the associated multiple processing algorithms to the GPU in the processing order, and allocate the corresponding running time of each processing algorithm based on the time ratio required to run each processing algorithm. In this way, refined virtualization processing of GPU resources according to the virtualization ratio is achieved, without having to rely heavily on the underlying GPU computing resource virtualization, greatly reducing the limitations of subsequent GPU resource scheduling, and can flexibly schedule GPU resources based on this configuration.

[0233] A flowchart of GPU resource scheduling provided by an embodiment of the present invention can be as follows: Figure 10 Shown, including:

[0234] S1001: Obtain pending tasks and start GPU resource scheduling.

[0235] The electronic device may obtain the task to be processed and start GPU resource scheduling to process the task to be processed.

[0236] S1002: Determine whether the target processing algorithm can be executed by the virtualized GPU.

[0237] The electronic device may determine whether the target processing algorithm for the task to be processed can be executed by the virtualized GPU. If so, step S1006 is executed; if not, step S1003 is executed. The electronic device may determine whether the processing algorithms corresponding to the virtualization technology pre-configured on the virtualized GPU include the target processing algorithm. If so, it may be determined that the target processing algorithm can be executed by the virtualized GPU.

[0238] S1003: Is there an idle physical GPU?

[0239] If the target processing algorithm of the task to be processed cannot be executed by the virtualized GPU, the electronic device may determine whether there is an idle physical GPU, and if so, execute step S1004; if not, execute step S1005.

[0240] S1004: Send the pending task to an idle physical GPU.

[0241] If there is an idle physical GPU, the electronic device may send the to-be-processed task to the idle physical GPU. After receiving the to-be-processed task, the idle physical GPU may start processing the to-be-processed task.

[0242] S1005: Temporarily do not process the pending tasks in the cache queue and wait for rescheduling.

[0243] If there is no idle physical GPU, the pending tasks in the cache queue may not be processed temporarily. When an idle physical GPU becomes available, the electronic device may schedule physical GPU resources again to process the pending tasks.

[0244] S1006: Determine whether there is an idle virtualized GPU that meets the scheduling requirements.

[0245] If the target processing algorithm of the pending task can be run by a virtualized GPU, the electronic device may determine whether there is an idle virtualized GPU in the resource pool that meets the scheduling requirements. If so, the electronic device executes step S1007; if not, the electronic device executes step S1005. The electronic device may determine whether there is an idle virtualized GPU in the resource pool based on the virtualization ratio of the virtualized GPUs in the resource pool and the occupied computing power of the virtualized GPUs.

[0246] S1007: Determine whether the idle virtualized GPUs meet the scheduling requirements.

[0247] If there is an idle virtualized GPU, the electronic device can determine whether the idle virtualized GPU meets the scheduling requirements of the pending task. If so, step S1008 is executed; if not, step S1005 is executed. Specifically, the electronic device can determine whether the remaining computing power of the idle virtualized GPU meets the computing power requirements of the pending task.

[0248] S1008: Determine whether there is a first virtualized GPU that supports the target physical layer virtualization technology.

[0249] If there is an idle virtualized GPU that meets the scheduling requirements, the electronic device can determine whether there is a first virtualized GPU that supports the target physical layer virtualization technology among the idle virtualized GPUs that meet the scheduling requirements. If so, execute step S1009; if not, execute step S1010.

[0250] S1009: Send the task to be processed to the first virtualized GPU.

[0251] If there is a first virtualized GPU that supports the target physical layer virtualization technology, the electronic device can send the task to be processed to the first virtualized GPU. After receiving the task to be processed, the first virtualized GPU can start processing the task to be processed.

[0252] S1010: Determine whether there is a second virtualized GPU that supports the target application layer virtualization technology.

[0253] If there is no first virtualized GPU that supports the target physical layer virtualization technology, the electronic device can determine whether there is a second virtualized GPU that supports the target application layer virtualization technology among the idle virtualized GPUs that meet the scheduling requirements. For the computing node with the second virtualized GPU, the electronic device can determine whether the second virtualized GPU corresponding to the computing node is running other processing algorithms that are pre-bound and associated with the target processing algorithm; if so, determine that the second virtualized GPU corresponding to the computing node is the first idle application layer virtualized GPU, and execute step S1011; if not, execute step S1012.

[0254] S1011: Sending the to-be-processed task to the first idle application layer virtualized GPU.

[0255] The electronic device may send the task to be processed to the first idle application layer virtualization GPU. After receiving the task to be processed, the first idle application layer virtualization GPU may process the task to be processed.

[0256] S1012: Send the task to be processed to the first idle application layer virtualized GPU of the other bound target processing algorithm.

[0257] If the electronic device determines that the second virtualized GPU corresponding to the above-mentioned computing node is not running other processing algorithms that are pre-bound and associated with the target processing algorithm, the electronic device can determine whether the second virtualized GPU corresponding to the next computing node is running other processing algorithms that are pre-bound and associated with the target processing algorithm, until it is determined that all second virtualized GPUs are not running other processing algorithms that are pre-bound and associated with the target processing algorithm, and determine the second virtualized GPU that supports the target processing algorithm as the first idle application layer virtualized GPU. Then, the task to be processed can be sent to the first idle application layer virtualized GPU. After receiving the task to be processed, the first idle application layer virtualized GPU can start processing the task to be processed.

[0258] S1013: GPU resource scheduling is completed.

[0259] In this embodiment, computing resources include physical GPUs and virtualized GPUs, and virtualized GPUs include physical layer virtualized GPUs and application layer virtualized GPUs. Therefore, the electronic device can allocate target computing resources for the tasks to be processed from the resource pool based on the target processing algorithm corresponding to the tasks to be processed and the idle status of the physical GPUs, physical layer virtualized GPUs, and application layer virtualized GPUs in the resource pool. It does not have to rely heavily on the virtualization of the underlying GPU computing resources. Even if the underlying GPU of the computing resources does not support physical layer virtualization, the combination of application layer algorithm scheduling and underlying virtualization technology can be used to enable a single GPU to run multiple algorithms simultaneously, greatly reducing the limitations of GPU resource scheduling and allowing flexible scheduling of GPU resources.

[0260] Corresponding to the above-mentioned GPU resource scheduling method, an embodiment of the present invention further provides a GPU resource scheduling device. The GPU resource scheduling device provided by an embodiment of the present invention is introduced below.

[0261] like Figure 11 As shown, a GPU resource scheduling device includes:

[0262] The pending task acquisition module 1101 is used to acquire pending tasks.

[0263] The resource allocation module 1102 is configured to allocate target computing resources from the resource pool to the task to be processed according to the target processing algorithm corresponding to the task to be processed and the idleness of computing resources in the resource pool.

[0264] The computing resources include physical GPUs and virtualized GPUs, and the virtualized GPUs include physical layer virtualized GPUs and application layer virtualized GPUs.

[0265] It can be seen that in the solution provided by the embodiment of the present invention, the electronic device can obtain the task to be processed, and then, according to the target processing algorithm corresponding to the task to be processed and the idle status of the computing resources in the resource pool, the target computing resources can be allocated from the resource pool to the task to be processed. Among them, the computing resources include physical GPUs and virtualized GPUs, and the virtualized GPUs include physical layer virtualized GPUs and application layer virtualized GPUs. In the above solution, the computing resources include physical GPUs and virtualized GPUs, and the virtualized GPUs include physical layer virtualized GPUs and application layer virtualized GPUs. Therefore, the electronic device can allocate the target computing resources from the resource pool to the task to be processed according to the target processing algorithm corresponding to the task to be processed and the idle status of the physical GPUs, physical layer virtualized GPUs, and application layer virtualized GPUs in the resource pool. It does not have to rely heavily on the virtualization of the underlying GPU computing resources. Even if the underlying GPU of the computing resources does not support physical layer virtualization, it can combine the algorithm scheduling of the application layer with the underlying virtualization technology to achieve the simultaneous operation of multiple algorithms on a single GPU, greatly reducing the limitations of GPU resource scheduling and enabling flexible scheduling of GPU resources.

[0266] As an implementation manner of the embodiment of the present invention, the above-mentioned device may further include:

[0267] The virtualization processing module is used to perform virtualization processing on the GPU in the resource pool to obtain the computing resources before obtaining the task to be processed.

[0268] Among them, the resource pool includes multiple GPUs that are preset to participate in resource scheduling; the proportion of the virtualized GPU in the computing resources of the resource pool meets the first proportion; and / or the proportion of the virtualized GPU of the target computing node in the resource pool in the computing resources of the target computing node meets the second proportion.

[0269] As an implementation of an embodiment of the present invention, the resource allocation module 1102 may include:

[0270] A first determining unit, configured to determine whether the target processing algorithm can be executed by a virtualized GPU;

[0271] a second determining unit, configured to determine whether there is an idle virtualized GPU in the resource pool that meets the scheduling requirements if the target processing algorithm can be executed by the virtualized GPU;

[0272] The target GPU determining unit is configured to determine a target virtualized GPU from the idle virtualized GPUs that meet the scheduling requirements.

[0273] The target virtualized GPU includes a first idle physical layer virtualized GPU or a first idle application layer virtualized GPU.

[0274] The task sending unit is used to send the to-be-processed task to the target virtualized GPU.

[0275] As an implementation manner of an embodiment of the present invention, the first judgment unit may include:

[0276] an algorithm determination subunit, configured to determine whether the processing algorithms corresponding to the virtualization technology pre-configured on the virtualized GPU include the target processing algorithm;

[0277] a determining subunit, configured to determine that the target processing algorithm can be executed by the virtualized GPU if the target processing algorithm is included;

[0278] As an implementation manner of an embodiment of the present invention, the second judgment unit may include:

[0279] an idle GPU determination subunit, configured to determine whether there is an idle virtualized GPU in the resource pool based on a virtualization ratio of the virtualized GPUs in the resource pool and an occupied computing power of the virtualized GPUs;

[0280] A computing power requirement determination subunit, configured to determine, if the idle virtualized GPU exists, whether the remaining computing power of the idle virtualized GPU meets the computing power requirement of the task to be processed;

[0281] an idle GPU determining subunit, configured to determine that the idle virtualized GPU is the idle virtualized GPU that meets the scheduling requirement if the computing power requirement of the task to be processed is met;

[0282] As an implementation manner of an embodiment of the present invention, the target GPU determination unit may include:

[0283] A first virtualized GPU determining subunit is configured to determine whether there is a first virtualized GPU that supports the target physical layer virtualization technology among the idle virtualized GPUs that meet the scheduling requirements;

[0284] The target physical layer virtualization technology is the physical layer virtualization technology applicable to the target processing algorithm.

[0285] The first target GPU determination subunit is configured to determine a first idle physical layer virtualization GPU from the first virtualized GPUs based on a priority of a physical layer virtualization technology corresponding to each first virtualized GPU if there are multiple first virtualized GPUs.

[0286] Among them, the priority of the physical layer virtualization technology is determined based on the processing performance of the simulated virtualized GPU corresponding to the first virtualized GPU. The simulated virtualized GPU is obtained by virtualizing the GPU corresponding to the first virtualized GPU through multiple physical layer virtualization technologies.

[0287] As an implementation manner of an embodiment of the present invention, the target GPU determination unit may further include:

[0288] A second virtualized GPU determining subunit is configured to determine whether there is a second virtualized GPU that supports the target application layer virtualization technology among the idle virtualized GPUs that meet the scheduling requirements;

[0289] The target application layer virtualization technology is an application layer virtualization technology applicable to the target processing algorithm.

[0290] The first target GPU determination subunit is configured to determine, for a computing node having the second virtualized GPU, whether the second virtualized GPU corresponding to the computing node is running other processing algorithms pre-bound and associated with the target processing algorithm; if so, determine that the second virtualized GPU corresponding to the computing node is the first idle application layer virtualized GPU; if not, determine whether the second virtualized GPU corresponding to the next computing node is running other processing algorithms pre-bound and associated with the target processing algorithm; until it is determined that all second virtualized GPUs are no longer running other processing algorithms pre-bound and associated with the target processing algorithm, determine the second virtualized GPU that supports the target processing algorithm as the first idle application layer virtualized GPU; or,

[0291] Used to, if there are multiple second virtualized GPUs in the resource pool, determine, for each second virtualized GPU, whether the second virtualized GPU is running other processing algorithms that are pre-bound and associated with the target processing algorithm; if so, determine that the second virtualized GPU is the first idle application layer virtualized GPU; if not, determine that the second virtualized GPU that supports the target processing algorithm is the first idle application layer virtualized GPU.

[0292] The embodiment of the present invention further provides an electronic device, such as Figure 12 As shown, it includes a processor 1201, a communication interface 1202, a memory 1203 and a communication bus 1204, wherein the processor 1201, the communication interface 1202, and the memory 1203 communicate with each other through the communication bus 1204.

[0293] Memory 1203, used for storing computer programs;

[0294] The processor 1201 is configured to implement the method steps described in any of the above embodiments when executing the program stored in the memory 1203 .

[0295] The communication bus mentioned in the electronic device mentioned above may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0296] The communication interface is used for communication between the above electronic device and other devices.

[0297] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.

[0298] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0299] In another embodiment of the present invention, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of the method described in any of the above embodiments are implemented.

[0300] In another embodiment of the present invention, a computer program product including instructions is provided, which, when executed on a computer, enables the computer to execute the method steps described in any one of the above embodiments.

[0301] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0302] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0303] Each embodiment in this specification is described in a related manner. Similar portions between embodiments can be referenced to each other. Each embodiment focuses on the differences between other embodiments. In particular, the device, electronic device, computer-readable storage medium, and computer program product embodiments are generally similar to the method embodiments, so their descriptions are relatively simple. For related portions, reference can be made to the descriptions of the method embodiments.

[0304] The above description is only a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention are included in the scope of protection of the present invention.

Claims

1. A GPU resource scheduling method, characterized in that: The method comprises: Get pending tasks; Determine whether the target processing algorithm corresponding to the task to be processed can be executed by the virtualized GPU; If the target processing algorithm can be run by a virtualized GPU, determining whether there is an idle virtualized GPU in a resource pool that meets the scheduling requirements, wherein the computing resources in the resource pool include physical GPUs and virtualized GPUs, and the virtualized GPUs include physical layer virtualized GPUs and application layer virtualized GPUs; Determine whether there is a first virtualized GPU that supports the target physical layer virtualization technology among the idle virtualized GPUs that meet the scheduling requirements; if there are multiple first virtualized GPUs, determine a first idle physical layer virtualized GPU from the first virtualized GPUs; or determine whether there is a second virtualized GPU that supports the target application layer virtualization technology among the idle virtualized GPUs that meet the scheduling requirements; if there is a second virtualized GPU, determine a first idle application layer virtualized GPU from the second virtualized GPUs, wherein the target physical layer virtualization technology is the physical layer virtualization technology applicable to the target processing algorithm, and the target application layer virtualization technology is the application layer virtualization technology applicable to the target processing algorithm; The to-be-processed task is sent to a target virtualized GPU, wherein the target virtualized GPU includes a first idle physical layer virtualized GPU or a first idle application layer virtualized GPU.

2. The method according to claim 1, characterized in that Before the step of obtaining the task to be processed, the method further includes: The GPUs in the resource pool are virtualized to obtain the computing resources, wherein the resource pool includes a plurality of preset GPUs participating in resource scheduling; the proportion of the virtualized GPU in the computing resources of the resource pool satisfies a first ratio; and / or the proportion of the virtualized GPU of the target computing node in the resource pool in the computing resources of the target computing node satisfies a second ratio.

3. The method according to claim 1, characterized in that The step of determining whether the target processing algorithm corresponding to the task to be processed can be executed by the virtualized GPU includes: Determining whether the processing algorithms corresponding to the virtualization technology pre-configured on the virtualized GPU include the target processing algorithm; If included, it is determined that the target processing algorithm can be executed by the virtualized GPU.

4. The method according to claim 1, wherein The step of determining whether there is an idle virtualized GPU in the resource pool that meets the scheduling requirements includes: Determining whether there is an idle virtualized GPU in the resource pool based on the virtualization ratio of the virtualized GPU in the resource pool and the occupied computing power of the virtualized GPU; If the idle virtualized GPU exists, determining whether the remaining computing power of the idle virtualized GPU meets the computing power requirement of the task to be processed; If so, it is determined that the idle virtualized GPU is the idle virtualized GPU that meets the scheduling requirement.

5. The method according to claim 1, wherein The step of determining a first idle physical layer virtualized GPU from the first virtualized GPU includes: Based on the priority of the physical layer virtualization technology corresponding to each first virtualized GPU, a first idle physical layer virtualization GPU is determined from the first virtualized GPUs, wherein the priority of the physical layer virtualization technology is determined based on the processing performance of the simulated virtualized GPU corresponding to the first virtualized GPU, and the simulated virtualized GPU is obtained by virtualizing the GPU corresponding to the first virtualized GPU through multiple physical layer virtualization technologies.

6. The method according to claim 1, wherein The step of determining the first idle application layer virtualized GPU from the second virtualized GPU includes: For a computing node having the second virtualized GPU, determine whether the second virtualized GPU corresponding to the computing node is running other processing algorithms pre-bound and associated with the target processing algorithm; if so, determine that the second virtualized GPU corresponding to the computing node is the first idle application layer virtualized GPU; if not, determine whether the second virtualized GPU corresponding to the next computing node is running other processing algorithms pre-bound and associated with the target processing algorithm; until it is determined that all second virtualized GPUs are no longer running other processing algorithms pre-bound and associated with the target processing algorithm, determine the second virtualized GPU that supports the target processing algorithm as the first idle application layer virtualized GPU; or, If there are multiple second virtualized GPUs in the resource pool, for each second virtualized GPU, determine whether the second virtualized GPU is running other processing algorithms that are pre-bound and associated with the target processing algorithm; if so, determine that the second virtualized GPU is the first idle application layer virtualized GPU; if not, determine that the second virtualized GPU that supports the target processing algorithm is the first idle application layer virtualized GPU.

7. A GPU resource scheduling device, characterized in that: The device comprises: A pending task acquisition module is used to obtain pending tasks; a resource allocation module, configured to allocate target computing resources from the resource pool to the task to be processed based on the target processing algorithm corresponding to the task to be processed and the idle status of computing resources in the resource pool; wherein the computing resources include physical GPUs and virtualized GPUs, and the virtualized GPUs include physical layer virtualized GPUs and application layer virtualized GPUs; The resource allocation module includes: A first determining unit, configured to determine whether the target processing algorithm can be executed by a virtualized GPU; a second determining unit, configured to determine whether there is an idle virtualized GPU in the resource pool that meets the scheduling requirements if the target processing algorithm can be executed by the virtualized GPU; a target GPU determining unit, configured to determine a target virtualized GPU from the idle virtualized GPUs that meet the scheduling requirements, wherein the target virtualized GPU includes a first idle physical layer virtualized GPU or a first idle application layer virtualized GPU; A task issuing unit, configured to issue the to-be-processed task to the target virtualized GPU; The target GPU determination unit is specifically used to determine whether there is a first virtualized GPU that supports the target physical layer virtualization technology among the idle virtualized GPUs that meet the scheduling requirements, and if there are multiple first virtualized GPUs, determine the first idle physical layer virtualization GPU from the first virtualized GPUs; or determine whether there is a second virtualized GPU that supports the target application layer virtualization technology among the idle virtualized GPUs that meet the scheduling requirements, and if the second virtualized GPU exists, determine the first idle application layer virtualization GPU from the second virtualized GPUs, wherein the target physical layer virtualization technology is the physical layer virtualization technology applicable to the target processing algorithm, and the target application layer virtualization technology is the application layer virtualization technology applicable to the target processing algorithm.

8. The device according to claim 7, characterized in that The device further comprises: a virtualization processing module configured to, before obtaining the task to be processed, perform virtualization processing on the GPU in the resource pool to obtain the computing resources, wherein the resource pool includes a plurality of preset GPUs participating in resource scheduling; the proportion of the virtualized GPU in the computing resources of the resource pool satisfies a first ratio; and / or the proportion of the virtualized GPU of the target computing node in the resource pool in the computing resources of the target computing node satisfies a second ratio; The first judgment unit includes: an algorithm determination subunit, configured to determine whether the processing algorithms corresponding to the virtualization technology pre-configured on the virtualized GPU include the target processing algorithm; a determining subunit, configured to determine that the target processing algorithm can be executed by the virtualized GPU if the target processing algorithm is included; The second judgment unit includes: an idle GPU determination subunit, configured to determine whether there is an idle virtualized GPU in the resource pool based on a virtualization ratio of the virtualized GPUs in the resource pool and an occupied computing power of the virtualized GPUs; A computing power requirement determination subunit, configured to determine, if the idle virtualized GPU exists, whether the remaining computing power of the idle virtualized GPU meets the computing power requirement of the task to be processed; an idle GPU determining subunit, configured to determine that the idle virtualized GPU is the idle virtualized GPU that meets the scheduling requirement if the computing power requirement of the task to be processed is met; The target GPU determination unit includes: a first virtualized GPU determination subunit, configured to determine whether there is a first virtualized GPU that supports a target physical layer virtualization technology among the idle virtualized GPUs that meet the scheduling requirements, wherein the target physical layer virtualization technology is a physical layer virtualization technology applicable to the target processing algorithm; a first target GPU determination subunit configured to determine, if there are multiple first virtualized GPUs, a first idle physical layer virtualization GPU from the first virtualized GPUs based on a priority of a physical layer virtualization technology corresponding to each first virtualized GPU, wherein the priority of the physical layer virtualization technology is determined based on a processing performance of a simulated virtualized GPU corresponding to the first virtualized GPU, where the simulated virtualized GPU is obtained by virtualizing the GPU corresponding to the first virtualized GPU using multiple physical layer virtualization technologies; The target GPU determination unit further includes: a second virtualized GPU determining subunit, configured to determine whether there is a second virtualized GPU that supports a target application layer virtualization technology among the idle virtualized GPUs that meet the scheduling requirements, wherein the target application layer virtualization technology is an application layer virtualization technology applicable to the target processing algorithm; The first target GPU determination subunit is configured to determine, for a computing node having the second virtualized GPU, whether the second virtualized GPU corresponding to the computing node is running other processing algorithms pre-bound and associated with the target processing algorithm; if so, determine that the second virtualized GPU corresponding to the computing node is the first idle application layer virtualized GPU; if not, determine whether the second virtualized GPU corresponding to the next computing node is running other processing algorithms pre-bound and associated with the target processing algorithm; until it is determined that all second virtualized GPUs are no longer running other processing algorithms pre-bound and associated with the target processing algorithm, determine the second virtualized GPU that supports the target processing algorithm as the first idle application layer virtualized GPU; or, Used to, if there are multiple second virtualized GPUs in the resource pool, determine, for each second virtualized GPU, whether the second virtualized GPU is running other processing algorithms that are pre-bound and associated with the target processing algorithm; if so, determine that the second virtualized GPU is the first idle application layer virtualized GPU; if not, determine that the second virtualized GPU that supports the target processing algorithm is the first idle application layer virtualized GPU.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory for storing computer programs; A processor, configured to implement the method steps described in any one of claims 1 to 6 when executing a program stored in a memory.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps of any one of claims 1 to 6 are implemented.

Images