Virtualization method, device and equipment of graphic processing unit (GPU) in container and storage medium

Abstract

A method, device and storage medium for virtualizing a graphics processing unit (GPU) in a container are provided. When a GPU resource request submitted by a first container is detected, a monitoring thread corresponding to the first container is activated. When a start operation for an application in the first container is detected, the monitoring thread monitors a calling command of the first container for a first GPU. Based on the calling command, a first GPU computing resource and a first GPU memory capacity are determined. When the first GPU memory capacity is less than or equal to a memory quota corresponding to the first container, the first GPU computing resource and the first GPU memory capacity are sent to a driver corresponding to the first GPU. The first GPU computing resource and the first GPU memory capacity are allocated to the first container by the driver corresponding to the first GPU. Thus, the GPU computing resource and the GPU memory capacity can be flexibly managed.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, device and storage medium for virtualizing a graphics processing unit (GPU) in a container. Background Technology

[0002] With the rapid development of fields such as artificial intelligence and machine learning, the demand for graphics processing unit (GPU) computing power is growing exponentially. This demand has driven the development of GPU container virtualization technology to utilize hardware resources more efficiently. GPU container virtualization technology encapsulates GPU resources into containers, achieving resource sharing and isolation. Compared to traditional virtualization technologies, GPU container virtualization technology offers higher performance and flexibility, supporting multi-user isolation scheduling strategies for the rational allocation of computing power. However, the GPU container virtualization method cannot fully meet the needs of users who require dynamic allocation and adjustment of GPU memory and computing power resources.

[0003] Therefore, how to achieve flexible management of GPU memory capacity and computing resources has become an urgent problem to be solved. Summary of the Invention

[0004] This application provides a method, apparatus, device, and storage medium for virtualizing a graphics processing unit (GPU) in a container, which enables flexible management of GPU memory capacity and computing resources.

[0005] In a first aspect, embodiments of this application provide a method for virtualizing a graphics processing unit (GPU) in a container, the method comprising:

[0006] Upon detecting a GPU resource request submitted through the first container, the monitoring thread corresponding to the first container is activated; the GPU resource request includes the first GPU computing power resources and the first GPU memory capacity requested for the first GPU.

[0007] Upon detecting a startup operation targeting an application within the first container, the monitoring thread monitors the invocation commands of the first container targeting the first GPU; the invocation commands are determined by the first container based on GPU resource requests; the invocation commands are used to access the first GPU's first GPU computing power resources and first GPU memory capacity;

[0008] Based on the command invocation, the computing power resources and memory capacity of the first GPU are determined.

[0009] If it is determined that the first GPU memory capacity is less than or equal to the memory quota corresponding to the first container, the first GPU computing power resources and the first GPU memory capacity are sent to the driver corresponding to the first GPU.

[0010] The first GPU computing power resources and first GPU memory capacity are allocated to the first container through the driver corresponding to the first GPU.

[0011] In one embodiment, the method further includes: determining the first GPU corresponding to the first container and the second GPU corresponding to the second container when it is determined that the GPU memory capacity of the first GPU called by the first container is less than or equal to the memory quota corresponding to the first container, and the second GPU memory capacity of the second GPU called by the second container is less than or equal to the memory quota corresponding to the second container; determining the target GPU computing power resources based on the computing power resources of the first GPU and the second GPU called by the second container when it is determined that the first GPU and the second GPU are the same; and determining the target GPU memory capacity based on the first GPU memory capacity and the second GPU memory capacity; sending the target GPU computing power resources and the target GPU memory capacity to the driver corresponding to the first GPU or the second GPU; and allocating the target GPU computing power resources and the target GPU memory capacity to the first container and the second container through the driver corresponding to the first GPU or the second GPU.

[0012] In one embodiment, the method further includes: when it is determined that the first GPU and the second GPU are different, sending the computing power resources and the memory capacity of the first GPU to the driver of the first GPU, and sending the computing power resources and the memory capacity of the second GPU to the driver of the second GPU; allocating the computing power resources and the memory capacity of the first GPU to the first container through the driver of the first GPU, and allocating the computing power resources and the memory capacity of the second GPU to the second container through the driver of the second GPU.

[0013] In one embodiment, the method further includes: performing time-slicing processing on the computing tasks in the first container and the computing tasks in the second container respectively to obtain multiple first time-slice tasks corresponding to the first container and multiple second time-slice tasks corresponding to the second container; if the first GPU and the second GPU are determined to be the same, rearranging the multiple first time-slice tasks and the multiple second time-slice tasks based on the computing power resources of the first GPU and the second GPU to obtain rearranged multiple time-slice tasks; wherein the time-slice ratio of the multiple first time-slice tasks and the multiple second time-slice tasks is the same as the ratio of GPU computing power resources called by each container; sending the rearranged multiple time-slice tasks to the driver of the first GPU or the second GPU so that the driver of the first GPU or the second GPU executes the rearranged multiple time-slice tasks.

[0014] In one embodiment, the method further includes: when it is determined that the first GPU and the second GPU are different, sending a plurality of first time slice tasks to the driver of the first GPU so that the driver of the first GPU executes the plurality of first time slice tasks; and sending a plurality of second time slice tasks to the driver of the second GPU so that the driver of the second GPU executes the plurality of second time slice tasks.

[0015] In one embodiment, the GPU resource request further includes indication information for obtaining additional GPU resources; the method further includes: monitoring the usage information of GPU computing power resources and GPU memory capacity corresponding to the first container through a monitoring thread; when it is determined that the GPU resources required by the first container exceed the GPU resources requested by the first container, and when it is determined that there are idle GPU resources in the GPU, additional GPU resources are allocated to the first container; the GPU resources include GPU computing power resources and GPU memory capacity.

[0016] Secondly, this application provides a virtualization apparatus for a graphics processing unit (GPU) in a container, the apparatus comprising:

[0017] The processing module is used to activate the monitoring thread corresponding to the first container when a GPU resource request submitted through the first container is detected; the GPU resource request includes the first GPU computing power resources and the first GPU memory capacity of the requested first GPU;

[0018] The monitoring module is used to monitor, via a monitoring thread, the invocation commands of the first container to the first GPU when a startup operation for an application in the first container is detected; the invocation commands are determined by the first container based on GPU resource requests; the invocation commands are used to access the first GPU's first GPU computing power resources and first GPU memory capacity;

[0019] The determination module is used to determine the first GPU computing power resources and the first GPU memory capacity based on the call command;

[0020] The processing module is used to send the computing power resources and the memory capacity of the first GPU to the driver corresponding to the first GPU when it is determined that the memory capacity of the first GPU is less than or equal to the memory quota corresponding to the first container.

[0021] The allocation module is used to allocate the first GPU computing power resources and the first GPU memory capacity to the first container through the driver corresponding to the first GPU.

[0022] Thirdly, this application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0023] Upon detecting a GPU resource request submitted through the first container, the monitoring thread corresponding to the first container is activated; the GPU resource request includes the first GPU computing power resources and the first GPU memory capacity requested for the first GPU;

[0024] Upon detecting a startup operation targeting an application within the first container, the monitoring thread monitors the invocation commands of the first container targeting the first GPU; the invocation commands are determined by the first container based on GPU resource requests; the invocation commands are used to access the first GPU's first GPU computing power resources and first GPU memory capacity;

[0025] Based on the command invocation, the computing power resources and memory capacity of the first GPU are determined.

[0026] If it is determined that the first GPU memory capacity is less than or equal to the memory quota corresponding to the first container, the first GPU computing power resources and the first GPU memory capacity are sent to the driver corresponding to the first GPU.

[0027] The first GPU computing power resources and first GPU memory capacity are allocated to the first container through the driver corresponding to the first GPU.

[0028] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0029] Upon detecting a GPU resource request submitted through the first container, the monitoring thread corresponding to the first container is activated; the GPU resource request includes the first GPU computing power resources and the first GPU memory capacity requested for the first GPU;

[0030] Upon detecting a startup operation targeting an application within the first container, the monitoring thread monitors the invocation commands of the first container targeting the first GPU; the invocation commands are determined by the first container based on GPU resource requests; the invocation commands are used to access the first GPU's first GPU computing power resources and first GPU memory capacity;

[0031] Based on the command invocation, the computing power resources and memory capacity of the first GPU are determined.

[0032] If it is determined that the first GPU memory capacity is less than or equal to the memory quota corresponding to the first container, the first GPU computing power resources and the first GPU memory capacity are sent to the driver corresponding to the first GPU.

[0033] The first GPU computing power resources and first GPU memory capacity are allocated to the first container through the driver corresponding to the first GPU.

[0034] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0035] Upon detecting a GPU resource request submitted through the first container, the monitoring thread corresponding to the first container is activated; the GPU resource request includes the first GPU computing power resources and the first GPU memory capacity requested for the first GPU.

[0036] Upon detecting a startup operation targeting an application within the first container, the monitoring thread monitors the invocation commands of the first container targeting the first GPU; the invocation commands are determined by the first container based on GPU resource requests; the invocation commands are used to access the first GPU's first GPU computing power resources and first GPU memory capacity;

[0037] Based on the command invocation, the computing power resources and memory capacity of the first GPU are determined.

[0038] If it is determined that the first GPU memory capacity is less than or equal to the memory quota corresponding to the first container, the first GPU computing power resources and the first GPU memory capacity are sent to the driver corresponding to the first GPU.

[0039] The first GPU computing power resources and first GPU memory capacity are allocated to the first container through the driver corresponding to the first GPU.

[0040] The aforementioned method, apparatus, device, and storage medium for virtualizing a graphics processing unit (GPU) in a container allow a computer device to activate a monitoring thread corresponding to the first container upon detecting a GPU resource request submitted through the first container. The GPU resource request includes the requested first GPU computing power resources and first GPU memory capacity of the first GPU. Upon detecting a startup operation for an application within the first container, the monitoring thread monitors the call commands from the first container to the first GPU. The call commands are determined by the first container based on the GPU resource request. The call commands are used to access the first GPU computing power resources and first GPU memory capacity of the first GPU. Based on the call commands, the first GPU computing power resources and first GPU memory capacity are determined. If the first GPU memory capacity is determined to be less than or equal to the memory quota corresponding to the first container, the first GPU computing power resources and first GPU memory capacity are sent to the driver corresponding to the first GPU. The driver corresponding to the first GPU allocates the first GPU computing power resources and first GPU memory capacity to the first container. By using this method, computer devices can monitor the GPU calls made by containers to determine the GPU computing power resources and GPU memory capacity that the container wants to call. If the GPU memory capacity to be called by the container is less than or equal to the memory quota corresponding to the container, the corresponding GPU computing power resources and GPU memory capacity are allocated to the container through the GPU driver. In this way, computer devices can flexibly manage the GPU resources (i.e., GPU computing power resources and GPU memory capacity) in the GPU according to actual needs, thereby improving the utilization rate of GPU resources and reducing the waste of GPU resources. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a schematic diagram illustrating an application scenario of a GPU virtualization method in a container, as provided in an embodiment of this application.

[0043] Figure 2 This is a flowchart illustrating a GPU virtualization method in a container provided in an embodiment of this application;

[0044] Figure 3 This is a flowchart illustrating another GPU virtualization method in a container provided in an embodiment of this application;

[0045] Figure 4This is a schematic diagram of yet another GPU virtualization method in a container provided in this application embodiment;

[0046] Figure 5 This is a schematic diagram of the structure of a graphics processing unit (GPU) virtualization device in a container, as provided in an embodiment of this application.

[0047] Figure 6 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0049] Please see Figure 1 , Figure 1 This is a schematic diagram illustrating an application scenario of a GPU virtualization method in a container, as provided in an embodiment of this application. For example... Figure 1 As shown, the computer device 100 includes a graphics processing unit (GPU) 101.

[0050] Computer device 100 can activate the monitoring thread corresponding to the first container upon detecting a GPU resource request submitted through the first container; the GPU resource request includes the requested first GPU computing power resources and first GPU memory capacity of the first GPU; upon detecting a startup operation for an application in the first container, the monitoring thread monitors the call commands of the first container for the first GPU; the call commands are determined by the first container based on the GPU resource request; the call commands are used to call the first GPU computing power resources and first GPU memory capacity of the first GPU; based on the call commands, the first GPU computing power resources and first GPU memory capacity are determined; if it is determined that the first GPU memory capacity is less than or equal to the memory quota corresponding to the first container, the first GPU computing power resources and first GPU memory capacity are sent to the driver corresponding to the first GPU; the driver corresponding to the first GPU allocates the first GPU computing power resources and first GPU memory capacity to the first container. By using this method, computer devices can monitor the GPU calls made by containers to determine the GPU computing power resources and GPU memory capacity that the container wants to call. If the GPU memory capacity to be called by the container is less than or equal to the memory quota corresponding to the container, the corresponding GPU computing power resources and GPU memory capacity are allocated to the container through the GPU driver. In this way, computer devices can flexibly manage the GPU resources (i.e., GPU computing power resources and GPU memory capacity) in the GPU according to actual needs, thereby improving the utilization rate of GPU resources and reducing the waste of GPU resources.

[0051] Optionally, computer device 100 may be a server. The server mentioned here may be a standalone physical server, a server cluster consisting of multiple physical servers, or a distributed system, etc.

[0052] Please see Figure 2 , Figure 2 This is a flowchart illustrating a GPU virtualization method in a container, as provided in an embodiment of this application. The method can be executed by a computer device (e.g., the computer device 100 described above). Figure 2 As shown, the virtualization method of the GPU in the container may include, but is not limited to, the following steps:

[0053] S201. Upon detecting a GPU resource request submitted through the first container, activate the monitoring thread corresponding to the first container; the GPU resource request includes the first GPU computing power resources and the first GPU memory capacity of the requested first GPU.

[0054] GPU resource requests can be submitted by the user in the first container displayed in the user interface.

[0055] The monitoring thread corresponding to the first container can be used to track commands that call GPU resources within the first container and continuously monitor the usage of GPU resources. This ensures that resource usage conforms to the established allocation strategy and monitoring requirements.

[0056] For example, the computer device may activate the monitoring thread corresponding to container 1 upon detecting a GPU resource request submitted by a user through container 1; wherein the GPU resource request includes GPU computing power resources (e.g., 80%) and GPU memory capacity (e.g., 1GB) requested by container 1 for GPU 1. The computer device may also activate the monitoring thread corresponding to container 2 upon detecting a GPU resource request submitted by a user through container 2.

[0057] S202. Upon detecting a startup operation for an application in the first container, monitor the call commands from the first container to the first GPU via a monitoring thread.

[0058] The invocation command is determined by the first container based on the GPU resource request; the invocation command is used to invoke the first GPU computing power resources and the first GPU memory capacity of the first GPU.

[0059] In one alternative implementation, when a computer device detects a startup operation for an application in a first container, it monitors the call commands of the first container for a first GPU via a monitoring thread. This may include: determining the call commands for the first GPU based on GPU resource requests through the first container; and, upon detecting a startup operation for an application in the first container, sending the call commands to the virtual GPU through the first container and monitoring the call commands via the monitoring thread.

[0060] For example, the computer device can first determine the call command for GPU1 based on the GPU resource request through container 1; the corresponding monitoring thread monitors the call command for the first GPU through container 1; then, when a startup operation for application (APP) 1 in container 1 is detected, the call command is sent to the virtual GPU through container 1, and the call command is monitored through the monitoring thread.

[0061] S203. Based on the call command, determine the computing power resources of the first GPU and the memory capacity of the first GPU.

[0062] The information included in the GPU resource request is the same as that included in the invocation command. The difference between the GPU resource request and the invocation command is that the GPU resource request is made by the user through the first container to request GPU resources, while the invocation command is used by the first container to invoke GPU resources.

[0063] Since the invocation command is determined by the first container based on the GPU resource request, and the GPU resource request includes the first GPU computing power resources and the first GPU memory capacity of the requested first GPU, the computer device can determine the first GPU computing power resources and the first GPU memory capacity based on the invocation command.

[0064] In one alternative implementation, the computer device determines the first GPU computing power resources and the first GPU memory capacity based on a call command. This can be achieved through a monitoring thread, which determines the first GPU computing power resources and the first GPU memory capacity based on the call command.

[0065] S204. If it is determined that the first GPU memory capacity is less than or equal to the memory quota corresponding to the first container, the first GPU computing power resources and the first GPU memory capacity are sent to the driver corresponding to the first GPU.

[0066] The video memory quota corresponding to the first container is declared when the first container is created.

[0067] In one alternative implementation, when the computer device determines that the first GPU memory capacity is less than or equal to the memory quota corresponding to the first container, it sends the first GPU computing power resources and the first GPU memory capacity to the driver corresponding to the first GPU. This can be done through a monitoring thread, which sends the first GPU computing power resources and the first GPU memory capacity to the driver corresponding to the first GPU when it is determined that the first GPU memory capacity is less than or equal to the memory quota corresponding to the first container.

[0068] In other words, if a computer device determines, through a monitoring thread, that the GPU memory capacity called by the first container is less than or equal to the memory quota corresponding to the first container, it can request GPU resources from the driver corresponding to the first GPU.

[0069] For example, suppose that the GPU memory capacity called by container 1 is 1GB and the corresponding memory quota of container 1 is 1.2GB. In this case, the computer device can determine that the GPU memory capacity called by container 1 is 1GB, which is less than the corresponding memory quota of container 1, 1.2GB. At this time, the computer device can send the GPU resources (GPU computing power resources and GPU memory capacity) called by container 1 to the driver corresponding to the GPU called by container 1.

[0070] In an optional implementation, the computer device may further output indication information if, through a monitoring thread, it is determined that the first GPU memory capacity is greater than the memory quota corresponding to the first container. This indication information indicates that the GPU memory capacity used by the first container is greater than the memory quota corresponding to the first container. Optionally, this indication information may be an ERR_MEM_MALLOC error signal.

[0071] For example, suppose that the GPU memory capacity called by container 2 is 1GB and the corresponding memory quota of container 2 is 0.8GB. In this case, the computer device can determine that the GPU memory capacity called by container 2 is 1GB, which is greater than the memory quota of container 1 of 0.8GB. At this time, the computer device can output the ERR_MEM_MALLOC error signal through the monitoring thread corresponding to container 2.

[0072] S205. Allocate first GPU computing power resources and first GPU memory capacity to the first container through the driver corresponding to the first GPU.

[0073] In other words, if the computer device determines that the first GPU memory capacity is less than or equal to the memory quota corresponding to the first container, it can allocate the first GPU computing power resources and the first GPU memory capacity to the first container through the memory corresponding to the first GPU.

[0074] For example, assuming that the GPU computing power resources of GPU1 called by container 1 are 80% and the GPU memory capacity is 1GB, the computer device can allocate 80% of the GPU computing power resources and 1GB of memory capacity of GPU1 to container 1 through the driver corresponding to GPU1, even if the GPU memory capacity called by container 1 is less than the memory quota of container 1.

[0075] In this embodiment, the computer device can activate the monitoring thread corresponding to the first container when it detects a GPU resource request submitted through the first container. The GPU resource request includes the first GPU computing power resources and the first GPU memory capacity of the requested first GPU. When a startup operation for an application in the first container is detected, the monitoring thread monitors the call commands of the first container for the first GPU. The call commands are determined by the first container based on the GPU resource request. The call commands are used to call the first GPU computing power resources and the first GPU memory capacity of the first GPU. Based on the call commands, the first GPU computing power resources and the first GPU memory capacity are determined. If it is determined that the first GPU memory capacity is less than or equal to the memory quota corresponding to the first container, the first GPU computing power resources and the first GPU memory capacity are sent to the driver corresponding to the first GPU. The driver corresponding to the first GPU allocates the first GPU computing power resources and the first GPU memory capacity to the first container. By using this method, computer devices can monitor the GPU calls made by containers to determine the GPU computing power resources and GPU memory capacity that the container wants to call. If the GPU memory capacity to be called by the container is less than or equal to the memory quota corresponding to the container, the corresponding GPU computing power resources and GPU memory capacity are allocated to the container through the GPU driver. In this way, computer devices can flexibly manage the GPU resources (i.e., GPU computing power resources and GPU memory capacity) in the GPU according to actual needs, thereby improving the utilization rate of GPU resources and reducing the waste of GPU resources.

[0076] Please see Figure 3 , Figure 3 This is a flowchart illustrating another GPU virtualization method in a container provided in this application embodiment. Figure 2 Compared to the GPU virtualization method shown in the container, Figure 3 The method illustrated also explains how a computer device sends the GPU resources requested by different containers to the corresponding GPU driver when different containers simultaneously access GPU resources. For example... Figure 3 As shown, the method may include, but is not limited to, the following steps:

[0077] S301. If a first GPU resource request submitted through the first container is detected, activate the first monitoring thread corresponding to the first container; and if a second GPU resource request submitted through the second container is detected, activate the second monitoring thread corresponding to the second container.

[0078] The first GPU resource request includes the first GPU computing power resources and the first GPU memory capacity requested for the first GPU of the first container; the second GPU resource request includes the first GPU computing power resources and the second GPU memory capacity requested for the second GPU of the second container.

[0079] Optionally, the smallest allocation unit for GPU computing resources can be 1%, meaning that the GPU computing resources requested for the first container or the second container can be any one of 1%, 2%, 3%, ..., 100%.

[0080] S302. If a startup operation for an application in the first container is detected, the first call command of the first container to the first GPU is monitored by the first monitoring thread; and if a startup operation for an application in the second container is detected, the first call command of the second container to the second GPU is monitored by the second monitoring thread.

[0081] The first invocation command is determined by the first container based on the first GPU resource request, and the first invocation command is used to call the first GPU computing power resources and the first GPU memory capacity of the first GPU; the second invocation command is determined by the second container based on the second GPU resource request, and the second invocation command is used to call the second GPU computing power resources and the second GPU memory capacity of the second GPU.

[0082] S303. Based on the first call command, determine the first GPU computing power resources and the first GPU memory capacity, and based on the second call command, determine the second GPU computing power resources and the second GPU memory capacity.

[0083] In an optional implementation, the relevant descriptions of steps S301 to S303 can be referred to the descriptions of steps S201 to S203 above, and will not be repeated here.

[0084] S304. If it is determined that the GPU memory capacity of the first GPU called by the first container is less than or equal to the memory quota corresponding to the first container, and the GPU memory capacity of the second GPU called by the second container is less than or equal to the memory quota corresponding to the second container, then determine the first GPU corresponding to the first container and the second GPU corresponding to the second container.

[0085] The video memory quota corresponding to the first container is declared when the first container is created; the video memory quota corresponding to the second container is declared when the second container is created.

[0086] For example, suppose that the GPU memory capacity called by container 1 is 1GB and the corresponding memory quota of container 1 is 1.2GB, the GPU memory capacity called by container 2 is 1GB and the corresponding memory quota of container 2 is 0.8GB. In this case, the computer device can determine the GPU resource to be called by container 1 and the GPU resource to be called by container 2.

[0087] S305. If it is determined that the first GPU and the second GPU are the same, the target GPU computing power resources are determined based on the computing power resources of the first GPU and the computing power resources of the second GPU called by the second container; and the target GPU memory capacity is determined based on the memory capacity of the first GPU and the memory capacity of the second GPU.

[0088] In one optional implementation, the computer device determines the target GPU computing resources based on the first GPU computing resources and the second GPU computing resources called by the second container. This can be achieved by summing the first GPU computing resources called by the first container and the second GPU computing resources called by the second container to obtain the total GPU computing resources; and using the total GPU computing resources as the target GPU computing resources.

[0089] Following the example in step S304 above, assuming that GPU1 (the GPU where the GPU resources to be called by container 1 are located) and GPU2 (the GPU where the GPU resources to be called by container 2 are located) are the same, and assuming that container 1 needs to call 50% of the GPU computing power resources of GPU1 and container 2 needs to call 30% of the GPU computing power resources of GPU2, in this case, the computer device can use the sum of 50% of the GPU computing power resources to be called by container 1 and 30% of the GPU computing power resources to be called by container 2, that is, 80% of the GPU computing power resources, as the target GPUS computing power resources.

[0090] In one optional implementation, the computer device determines the target GPU memory capacity based on the first GPU memory capacity and the second GPU memory capacity. This can be achieved by summing the first GPU memory capacity called by the first container and the second GPU memory capacity called by the second container to obtain the total GPU memory capacity. The total GPU memory capacity is then used as the target GPU computing power resource.

[0091] Following the example in step S304 above, assuming that GPU1 (the GPU where the GPU resources to be called by container 1 are located) and GPU2 (the GPU where the GPU resources to be called by container 2 are located) are the same, and assuming that the GPU memory capacity to be called by container 1 is 1GB and the GPU memory capacity to be called by container 2 is 0.8GB, in this case, the computer device can use the sum of the GPU memory capacity to be called by container 1 (1GB) and the GPU memory capacity to be called by container 2 (0.8GB), that is, 1.8GB, as the target GPU memory capacity.

[0092] S306. Send the target GPU computing power resources and target GPU memory capacity to the driver corresponding to the first GPU or the second GPU.

[0093] As mentioned above, the target CPU computing power and target GPU memory capacity are determined by the computer device assuming that the first GPU and the second GPU are identical. Therefore, the driver corresponding to the first GPU or the second GPU is the same driver. The computer device sends the target GPU computing power and target GPU memory capacity to the driver corresponding to the first GPU or the second GPU. In other words, the computer device can send the target GPU computing power and target GPU memory capacity to the target GPU, which is the GPU where the GPU resources to be accessed by container 1 and container 2 reside.

[0094] S307. Allocate target GPU computing power resources and target GPU memory capacity to the first container and the second container through the driver corresponding to the first GPU or the second GPU.

[0095] In this process, the computer device allocates target GPU computing power resources and target GPU memory capacity to the first container and the second container through the driver corresponding to the first GPU or the second GPU. This can also be understood as determining the target GPU computing power resources and target GPU memory capacity in the first GPU or the second GPU as GPU computing power resources and GPU memory capacity for executing computing tasks in the first container and the second container through the driver corresponding to the first GPU or the second GPU.

[0096] For example, assuming the target GPU computing power is 80% and the target GPU memory capacity is 1.8GB, the computer device can determine, through the driver of the first GPU or the second GPU, to use 80% of the GPU computing power and 1.8GB of GPU memory capacity in the first GPU or the second GPU as GPU resources for executing computing tasks in container 1 and computing tasks in container 2.

[0097] In this embodiment, the computer device can simultaneously call GPU resources in different containers, and when the GPU memory capacity called by different containers is less than or equal to the memory quota corresponding to the container, the GPU where the GPU resources called by different containers are located is determined; when it is determined that the GPU resources called by different containers are located on the same GPU, the GPU resources called by different containers are integrated to obtain the target GPU computing power resources and the target GPU memory capacity, and the target GPU computing power resources and the target GPU memory capacity are sent to the driver of the same GPU corresponding to different containers; through the driver of the GPU, the target GPU computing power resources and the target GPU memory capacity are allocated to different containers. In this way, on the one hand, by comparing the GPU memory capacity called by different containers with the corresponding memory quota of each container, and performing subsequent steps only when the GPU memory capacity called by different containers is less than or equal to the corresponding memory quota of each container, the compliance of memory capacity usage can be ensured, and over-allocation of memory capacity can be prevented. On the other hand, by integrating the GPU resources called by different containers and sending the integrated GPU resources to the driver of the same GPU corresponding to different containers, the multiple sending of GPU resources called by different containers to the driver of the same GPU can be avoided, thereby improving the efficiency of GPU resource allocation.

[0098] In an alternative implementation, after step S304, i.e. after the computer device determines the first GPU and the second GPU, the computer device may further perform the following steps:

[0099] Step 1: If it is determined that the first GPU and the second GPU are different, send the computing power resources and memory capacity of the first GPU to the driver of the first GPU, and send the computing power resources and memory capacity of the second GPU to the driver of the second GPU.

[0100] For example, suppose container 1 needs to use 50% of the GPU computing power and 1GB of GPU memory in GPU 1, and container 2 needs to use 30% of the GPU computing power and 0.5GB of GPU memory in GPU 2. In this case, the computer device will send 50% of the GPU computing power and 1GB of GPU memory to the driver of GPU 1, and send 30% of the GPU computing power and 0.5GB of GPU memory to the driver of GPU 2.

[0101] Step 2: Using the driver of the first GPU, allocate the first GPU computing power resources and the first GPU memory capacity to the first container; and using the driver of the second GPU, allocate the second GPU computing power resources and the second GPU memory capacity to the second container.

[0102] In other words, the computer device can use the driver of the first GPU to determine the first GPU computing power resources and the first GPU memory capacity in the first GPU as GPU resources for executing computing tasks in the first container, and use the driver of the second GPU to determine the second GPU computing power resources and the second GPU memory capacity in the second GPU as GPU resources for executing computing tasks in the second container.

[0103] Following the example above, the computer device can use the GPU1 driver to allocate 50% of the GPU computing power and 1GB of GPU memory in GPU1 as GPU resources for executing computing tasks in container 1, and use the GPU2 driver to allocate 30% of the GPU computing power and 0.5GB of GPU memory in GPU2 as GPU resources for executing computing tasks in container 2.

[0104] By adopting this implementation method, the computer device can send the GPU resources required by different containers to the driver of the GPU corresponding to the container, even when the GPU resources required by different containers are located on different GPUs. In this way, flexible management of GPU memory capacity and GPU computing power resources can be achieved.

[0105] In one alternative implementation, Figure 3 In the method shown, the computer device may also perform the following steps:

[0106] Step 1: Perform time-slicing processing on the computation tasks in the first container and the computation tasks in the second container respectively to obtain multiple first time-slice tasks corresponding to the first container and multiple second time-slice tasks corresponding to the second container.

[0107] Optionally, the minimum slice size can be 1MB, meaning that the size of each time slice task can be 1MB.

[0108] For example, the computer device can perform time-slicing processing on the computing tasks in the first container and the computing tasks in the second container in 2-millisecond units to obtain multiple first time-slice tasks corresponding to the first container and multiple second time-slice tasks corresponding to the second container.

[0109] In one alternative implementation, after the computer device performs step one, it may also perform steps two and three, or step four.

[0110] Step 2: Assuming the first GPU and the second GPU are identical, rearrange multiple first time slice tasks and multiple second time slice tasks based on the computing power resources of the first GPU and the second GPU to obtain multiple rearranged time slice tasks; wherein, the time slice ratio of each of the multiple first time slice tasks and multiple second time slice tasks is the same as the ratio of GPU computing power resources called by each container.

[0111] For example, assuming that the multiple first time slice tasks corresponding to container 1 are APP0_0, APP0_1, and APP0_2, and the multiple second time slice tasks corresponding to container 2 are APP1_0, APP1_1, and APP1_2, and assuming that the first GPU computing power resource is 80% and the second GPU computing power resource is 20%, then the computer device can, under the condition that the first GPU and the second GPU are the same, rearrange the multiple first time slice tasks and the multiple second time slice tasks based on the first GPU computing power resource and the second GPU computing power resource, to obtain the rearranged multiple time slice tasks as APP0_0, APP1_0, APP0_1, APP1_1, APP0_2, and APP1_2. The ratio of the time slices of the multiple first time slice tasks and the multiple second time slice tasks is the same as the ratio of the GPU computing power resources called by container 1 and container 2, which is 8:2.

[0112] Step 3: Send the orchestrated multiple time-slice tasks to the driver of the first GPU or the second GPU so that the driver of the first GPU or the second GPU can execute the orchestrated multiple time-slice tasks.

[0113] For example, following the example above, the computer device can send APP0_0, APP1_0, APP0_1, APP1_1, APP0_2, and APP1_2 to the driver of the first GPU or the second GPU, so that the first GPU or the second GPU first executes the computation task in container 1 for 8ms, and then executes the computation task in container 2 for 2ms. That is, the computation tasks in container 1 and the container are executed in the following order: execute APP0_0 task for 8ms, execute APP1_0 task for 2ms, execute APP0_1 task for 8ms, execute APP1_1 task for 2ms, execute APP0_2 task for 8ms, and execute APP1_2 task for 2ms.

[0114] Step 4: If it is determined that the first GPU and the second GPU are different, send multiple first time slice tasks to the driver of the first GPU so that the driver of the first GPU executes multiple first time slice tasks, and send multiple second time slice tasks to the driver of the second GPU so that the driver of the second GPU executes multiple second time slice tasks.

[0115] For example, assuming that the multiple first time slice tasks corresponding to container 1 are APP0_0, APP0_1 and APP0_2, and the multiple second time slice tasks corresponding to container 2 are APP1_0, APP1_1 and APP1_2, the computer device can send APP0_0, APP0_1 and APP0_2 to the driver of GPU1 and APP1_0, APP1_1 and APP1_2 to the driver of GPU2, even if the GPU resources called by container 1 are located on GPU1 and the GPU resources called by container 2 are located on GPU2.

[0116] In this implementation, the computer device can perform time-slicing processing on computing tasks in different containers to obtain multiple time-slice tasks. If the GPU resources called by different containers are located on the same GPU, the multiple time-slice tasks are rearranged and sent to the drivers of the GPUs where the GPU resources called by different containers are located. If the GPU resources called by different containers are located on different GPUs, the time-slice tasks corresponding to different containers are sent to the drivers of the GPUs where the GPU resources called by different containers are located. In this way, it can be ensured that computing tasks in different containers are executed simultaneously, thereby improving computing efficiency.

[0117] In one alternative implementation, Figure 1 The GPU resource request mentioned in the GPU virtualization method shown may also include instructions for obtaining additional GPU resources.

[0118] In this embodiment, the computer device may also perform the following steps:

[0119] Step 1: Monitor the usage information of GPU computing power resources and GPU memory capacity corresponding to the first container through the monitoring thread.

[0120] Step 2: If it is determined that the GPU resources required by the first container exceed the GPU resources requested by the first container, and it is determined that there are idle GPU resources in the GPU, allocate additional GPU resources to the first container.

[0121] GPU resources include GPU computing power and GPU memory capacity.

[0122] For example, suppose the GPU resources required by the first container include 1.5GB of video memory in GPU1, and the video memory requested by container 1 is 1GB of video memory in GPU1. Suppose there is also 1GB of free GPU video memory in GPU1. In this case, the computer device can allocate an additional 0.5GB of GPU video memory to container 1 through the driver of GPU1.

[0123] In one alternative implementation, the computer device may also release the additional GPU resources occupied by the first container after determining that the computational task in the first container has been completed. This further improves the flexibility of GPU resource management.

[0124] In this implementation, the computer device can monitor the usage information of GPU computing resources and GPU memory capacity corresponding to the first container through a monitoring thread. If it is determined that the GPU resources required by the first container exceed the GPU resources requested by the first container, and if it is determined that there are idle GPU resources available, additional GPU resources are allocated to the first container. Thus, even if the GPU resources used by the first container occasionally exceed the requested GPU resources, the computer device can still allocate additional GPU resources to the first container if it determines that there are idle GPU resources available, thereby avoiding the waste of idle GPU resources and improving the utilization rate of GPU resources. Furthermore, this implementation also improves the dynamic scalability of GPU resources.

[0125] The following example illustrates the GPU virtualization method in containers provided in this application, using the case where containers 1 and 2 utilize the same GPU resource. Please refer to [link to relevant documentation]. Figure 4 , Figure 4 This is a schematic diagram of another GPU virtualization method in a container provided in the embodiments of this application.

[0126] like Figure 4 As shown, this includes a computing power control unit (denoted as scheduler_control) and a memory control unit (denoted as memory_control). The scheduler_control is the allocation and management unit for the GPU's internal streaming multiprocessors (SM). The memory_control is the key control unit responsible for the management and allocation of the GPU's internal memory.

[0127] like Figure 4 As shown, a user can request 80% of the GPU computing power and 16GB of GPU memory through container 1, and 20% of the GPU computing power and 8GB of GPU memory through container 2; then, the user can start application 0 running in container 1 and application 1 running in container 2.

[0128] Then, the computer device can monitor the GPU call command 1 of container 1 through monitoring thread 1 corresponding to container 1. This call command 1 includes the GPU computing power resource 1 required by container 1 to be 80%, and the GPU memory capacity 1 to be 16GB. Similarly, the computer device can monitor the GPU call command 2 of container 2 through monitoring thread 2 corresponding to container 2. This call command 2 includes the GPU computing power resource 2 required by container 2 to be 20%, and the GPU memory capacity 2 to be 8GB. After determining the GPU computing power resource 1 called by container 1 through monitoring thread 1, it is determined whether this GPU computing power resource 1 exceeds the memory quota corresponding to container 1. If it does, an ERR_MEM_MALLOC error signal is sent to application 0; otherwise, the GPU memory capacity 1 is sent to memory_control. The computing task corresponding to application 0 is then processed by time slicing to obtain application 0_0, application 0_1, ..., application 0_n. After determining the GPU computing power resource 1 called by container 2 through monitoring thread 2, it is determined whether the GPU computing power resource 2 exceeds the memory quota corresponding to container 2; if it exceeds the quota, an ERR_MEM_MALLOC error signal is sent to application 1; otherwise, the GPU memory capacity 2 is sent to memory_control; the computing task corresponding to application 1 is processed by time slicing to obtain application 1_0, application 1_1, ..., application 1_n.

[0129] Next, the computer device can send application 0_0, application 0_1, ..., application 0_2, and application 1_0, application 1_1, ..., application 1_2 to scheduler_control, and scheduler_control will arrange the received multiple time-slice tasks to obtain multiple arranged time-slice tasks, namely application 0_0, application 1_0, application 0_1, and application 1_1, ..., application 1_n, ..., application 1_n; and memory_control will integrate GPU memory capacity 1 and GPU memory capacity 2.

[0130] Afterwards, the computer device can send the orchestrated multiple time-slice tasks and the integrated GPU memory capacity to the GPU driver, so that the GPU driver can allocate GPU resources to container 1 and container 2.

[0131] In this way, by isolating computing power and video memory at the driver layer, tasks can be ensured to run independently. Furthermore, it achieves complete transparency to user programs, ensuring that users do not experience any interference or changes during use, thus guaranteeing a smooth user experience when using the application.

[0132] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0133] Based on the same inventive concept, this application also provides a GPU virtualization apparatus for implementing the GPU virtualization method in a container described above. The solution provided by this apparatus is similar to the implementation described in the above method. Therefore, the specific limitations of the one or more GPU virtualization apparatus embodiments provided below can be found in the limitations of the GPU virtualization method in a container described above, and will not be repeated here.

[0134] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of a graphics processing unit (GPU) virtualization device in a container, as provided in an embodiment of this application. Figure 5 As shown, the virtualization device of the graphics processing unit (GPU) in the container may include, but is not limited to:

[0135] Processing module 501 is used to activate the monitoring thread corresponding to the first container when a GPU resource request submitted through the first container is detected; the GPU resource request includes the first GPU computing power resources and the first GPU memory capacity of the requested first GPU;

[0136] The monitoring module 502 is used to monitor, through a monitoring thread, the calling commands of the first container to the first GPU when a startup operation for an application in the first container is detected; the calling commands are determined by the first container based on GPU resource requests; the calling commands are used to call the first GPU computing power resources and the first GPU memory capacity of the first GPU;

[0137] The determination module 503 is used to determine the computing power resources of the first GPU and the memory capacity of the first GPU based on the call command;

[0138] The processing module 501 is used to send the computing power resources of the first GPU and the memory capacity of the first GPU to the driver corresponding to the first GPU when it is determined that the memory capacity of the first GPU is less than or equal to the memory quota corresponding to the first container.

[0139] The allocation module 504 is used to allocate the first GPU computing power resources and the first GPU memory capacity to the first container through the driver corresponding to the first GPU.

[0140] In one embodiment, the determining module 50 is further configured to: determine the first GPU corresponding to the first container and the second GPU corresponding to the second container when it is determined that the GPU memory capacity of the first GPU called by the first container is less than or equal to the memory quota corresponding to the first container, and the second GPU memory capacity of the second GPU called by the second container is less than or equal to the memory quota corresponding to the second container; when it is determined that the first GPU and the second GPU are the same, determine the target GPU computing power resources based on the computing power resources of the first GPU and the second GPU called by the second container; and determine the target GPU memory capacity based on the memory capacity of the first GPU and the memory capacity of the second GPU; the processing module 501 is further configured to: send the target GPU computing power resources and the target GPU memory capacity to the driver corresponding to the first GPU or the second GPU; and allocate the target GPU computing power resources and the target GPU memory capacity to the first container and the second container through the driver corresponding to the first GPU or the second GPU.

[0141] In one embodiment, the processing module 501 is further configured to: send the computing power resources and memory capacity of the first GPU to the driver of the first GPU when it is determined that the first GPU and the second GPU are different, and send the computing power resources and memory capacity of the second GPU to the driver of the second GPU; allocate the computing power resources and memory capacity of the first GPU to the first container through the driver of the first GPU, and allocate the computing power resources and memory capacity of the second GPU to the second container through the driver of the second GPU.

[0142] In one embodiment, the processing module 501 is further configured to: perform time-slicing processing on the computing tasks in the first container and the computing tasks in the second container respectively, to obtain multiple first time-slice tasks corresponding to the first container and multiple second time-slice tasks corresponding to the second container; if the first GPU and the second GPU are the same, rearrange the multiple first time-slice tasks and the multiple second time-slice tasks based on the computing power resources of the first GPU and the second GPU, to obtain rearranged multiple time-slice tasks; wherein the time-slice ratio of the multiple first time-slice tasks and the multiple second time-slice tasks is the same as the ratio of GPU computing power resources called by each container; and send the rearranged multiple time-slice tasks to the driver of the first GPU or the second GPU, so that the driver of the first GPU or the second GPU executes the rearranged multiple time-slice tasks.

[0143] In one embodiment, the processing module 501 is further configured to: send a plurality of first time slice tasks to the driver of the first GPU when it is determined that the first GPU and the second GPU are different, so that the driver of the first GPU executes the plurality of first time slice tasks; and send a plurality of second time slice tasks to the driver of the second GPU, so that the driver of the second GPU executes the plurality of second time slice tasks.

[0144] In one embodiment, the GPU resource request further includes an indication of obtaining additional GPU resources; the monitoring module 502 is also configured to monitor the usage information of GPU computing power resources and GPU memory capacity corresponding to the first container through a monitoring thread; the processing module 501 is also configured to allocate additional GPU resources to the first container when it is determined that the GPU resources required by the first container exceed the GPU resources requested by the first container, and when it is determined that there are idle GPU resources in the GPU; the GPU resources include GPU computing power resources and GPU memory capacity.

[0145] The various modules within the virtualization device of the aforementioned graphics processing unit (GPU) in the container can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independent of the processor in the terminal device, or stored in software within the memory of the terminal device, so that the processor can invoke and execute the operations corresponding to each module.

[0146] In one exemplary embodiment, this application provides a computer device, which may be a terminal, and its internal structure diagram may be as follows. Figure 6As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, and input devices. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface and input devices are connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When executed by the processor, the computer program implements a method for virtualizing a graphics processing unit (GPU) within a container.

[0147] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0148] In one exemplary embodiment, this application provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0149] Upon detecting a GPU resource request submitted through the first container, the monitoring thread corresponding to the first container is activated; the GPU resource request includes the first GPU computing power resources and the first GPU memory capacity requested for the first GPU;

[0150] Upon detecting a startup operation targeting an application within the first container, the monitoring thread monitors the invocation commands of the first container targeting the first GPU; the invocation commands are determined by the first container based on GPU resource requests; the invocation commands are used to access the first GPU's first GPU computing power resources and first GPU memory capacity;

[0151] Based on the command invocation, the computing power resources and memory capacity of the first GPU are determined.

[0152] If it is determined that the first GPU memory capacity is less than or equal to the memory quota corresponding to the first container, the first GPU computing power resources and the first GPU memory capacity are sent to the driver corresponding to the first GPU.

[0153] The first GPU computing power resources and first GPU memory capacity are allocated to the first container through the driver corresponding to the first GPU.

[0154] In one embodiment, the processor executing the computer program further performs the following steps: if it is determined that the GPU memory capacity of the first GPU called by the first container is less than or equal to the memory quota corresponding to the first container, and the GPU memory capacity of the second GPU called by the second container is less than or equal to the memory quota corresponding to the second container, determine the first GPU corresponding to the first container and the second GPU corresponding to the second container; if it is determined that the first GPU and the second GPU are the same, determine the target GPU computing power resources based on the computing power resources of the first GPU and the second GPU called by the second container; and determine the target GPU memory capacity based on the memory capacity of the first GPU and the memory capacity of the second GPU; send the target GPU computing power resources and the target GPU memory capacity to the driver corresponding to the first GPU or the second GPU; and allocate the target GPU computing power resources and the target GPU memory capacity to the first container and the second container through the driver corresponding to the first GPU or the second GPU.

[0155] In one embodiment, the processor executing the computer program further performs the following steps: when it is determined that the first GPU and the second GPU are different, sending the computing power resources and the memory capacity of the first GPU to the driver of the first GPU, and sending the computing power resources and the memory capacity of the second GPU to the driver of the second GPU; allocating the computing power resources and the memory capacity of the first GPU to the first container through the driver of the first GPU, and allocating the computing power resources and the memory capacity of the second GPU to the second container through the driver of the second GPU.

[0156] In one embodiment, the processor executing the computer program further performs the following steps: performing time-slicing processing on the computing tasks in the first container and the computing tasks in the second container respectively, to obtain multiple first time-slice tasks corresponding to the first container and multiple second time-slice tasks corresponding to the second container; if it is determined that the first GPU and the second GPU are the same, rearranging the multiple first time-slice tasks and the multiple second time-slice tasks based on the computing power resources of the first GPU and the second GPU, to obtain rearranged multiple time-slice tasks; wherein the time-slice ratio of the multiple first time-slice tasks and the multiple second time-slice tasks is the same as the ratio of GPU computing power resources called by each container; sending the rearranged multiple time-slice tasks to the driver of the first GPU or the second GPU, so that the driver of the first GPU or the second GPU executes the rearranged multiple time-slice tasks.

[0157] In one embodiment, when the processor executes the computer program, it further performs the following steps: if it is determined that the first GPU and the second GPU are different, it sends a plurality of first time slice tasks to the driver of the first GPU so that the driver of the first GPU executes the plurality of first time slice tasks; and it sends a plurality of second time slice tasks to the driver of the second GPU so that the driver of the second GPU executes the plurality of second time slice tasks.

[0158] In one embodiment, the GPU resource request further includes indication information for obtaining additional GPU resources; the processor executing the computer program also implements the following steps: monitoring the usage information of GPU computing power resources and GPU memory capacity corresponding to the first container through a monitoring thread; when it is determined that the GPU resources required by the first container exceed the GPU resources requested by the first container, and when it is determined that there are idle GPU resources in the GPU, allocating additional GPU resources to the first container; the GPU resources include GPU computing power resources and GPU memory capacity.

[0159] In one exemplary embodiment, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0160] Upon detecting a GPU resource request submitted through the first container, the monitoring thread corresponding to the first container is activated; the GPU resource request includes the first GPU computing power resources and the first GPU memory capacity requested for the first GPU;

[0161] Upon detecting a startup operation targeting an application within the first container, the monitoring thread monitors the invocation commands of the first container targeting the first GPU; the invocation commands are determined by the first container based on GPU resource requests; the invocation commands are used to access the first GPU's first GPU computing power resources and first GPU memory capacity;

[0162] Based on the command invocation, the computing power resources and memory capacity of the first GPU are determined.

[0163] If it is determined that the first GPU memory capacity is less than or equal to the memory quota corresponding to the first container, the first GPU computing power resources and the first GPU memory capacity are sent to the driver corresponding to the first GPU.

[0164] The first GPU computing power resources and first GPU memory capacity are allocated to the first container through the driver corresponding to the first GPU.

[0165] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: if it is determined that the GPU memory capacity of the first GPU called by the first container is less than or equal to the memory quota corresponding to the first container, and the GPU memory capacity of the second GPU called by the second container is less than or equal to the memory quota corresponding to the second container, determine the first GPU corresponding to the first container and the second GPU corresponding to the second container; if it is determined that the first GPU and the second GPU are the same, determine the target GPU computing power resources based on the computing power resources of the first GPU and the second GPU called by the second container; and determine the target GPU memory capacity based on the memory capacity of the first GPU and the memory capacity of the second GPU; send the target GPU computing power resources and the target GPU memory capacity to the driver corresponding to the first GPU or the second GPU; and allocate the target GPU computing power resources and the target GPU memory capacity to the first container and the second container through the driver corresponding to the first GPU or the second GPU.

[0166] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: if it is determined that the first GPU and the second GPU are different, sending the computing power resources and the memory capacity of the first GPU to the driver of the first GPU, and sending the computing power resources and the memory capacity of the second GPU to the driver of the second GPU; allocating the computing power resources and the memory capacity of the first GPU to the first container through the driver of the first GPU, and allocating the computing power resources and the memory capacity of the second GPU to the second container through the driver of the second GPU.

[0167] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: performing time-slicing processing on the computing tasks in the first container and the computing tasks in the second container respectively, to obtain multiple first time-slice tasks corresponding to the first container and multiple second time-slice tasks corresponding to the second container; if it is determined that the first GPU and the second GPU are the same, rearranging the multiple first time-slice tasks and the multiple second time-slice tasks based on the computing power resources of the first GPU and the second GPU, to obtain rearranged multiple time-slice tasks; wherein the time-slice ratio of the multiple first time-slice tasks and the multiple second time-slice tasks is the same as the ratio of GPU computing power resources called by each container; sending the rearranged multiple time-slice tasks to the driver of the first GPU or the second GPU, so that the driver of the first GPU or the second GPU executes the rearranged multiple time-slice tasks.

[0168] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: if it is determined that the first GPU and the second GPU are different, sending a plurality of first time slice tasks to the driver of the first GPU so that the driver of the first GPU executes the plurality of first time slice tasks, and sending a plurality of second time slice tasks to the driver of the second GPU so that the driver of the second GPU executes the plurality of second time slice tasks.

[0169] In one embodiment, the GPU resource request further includes indication information for obtaining additional GPU resources; when the computer program is executed by the processor, it also performs the following steps: monitoring the usage information of GPU computing power resources and GPU memory capacity corresponding to the first container through a monitoring thread; when it is determined that the GPU resources required by the first container exceed the GPU resources requested by the first container, and when it is determined that there are idle GPU resources in the GPU, additional GPU resources are allocated to the first container; GPU resources include GPU computing power resources and GPU memory capacity.

[0170] In one exemplary embodiment, this application provides a computer program product including a computer program that, when executed by a processor, performs the following steps:

[0171] Upon detecting a GPU resource request submitted through the first container, the monitoring thread corresponding to the first container is activated; the GPU resource request includes the first GPU computing power resources and the first GPU memory capacity requested for the first GPU;

[0172] Upon detecting a startup operation targeting an application within the first container, the monitoring thread monitors the invocation commands of the first container targeting the first GPU; the invocation commands are determined by the first container based on GPU resource requests; the invocation commands are used to access the first GPU's first GPU computing power resources and first GPU memory capacity;

[0173] Based on the command invocation, the computing power resources and memory capacity of the first GPU are determined.

[0174] If it is determined that the first GPU memory capacity is less than or equal to the memory quota corresponding to the first container, the first GPU computing power resources and the first GPU memory capacity are sent to the driver corresponding to the first GPU.

[0175] The first GPU computing power resources and first GPU memory capacity are allocated to the first container through the driver corresponding to the first GPU.

[0176] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: if it is determined that the GPU memory capacity of the first GPU called by the first container is less than or equal to the memory quota corresponding to the first container, and the GPU memory capacity of the second GPU called by the second container is less than or equal to the memory quota corresponding to the second container, determine the first GPU corresponding to the first container and the second GPU corresponding to the second container; if it is determined that the first GPU and the second GPU are the same, determine the target GPU computing power resources based on the computing power resources of the first GPU and the second GPU called by the second container; and determine the target GPU memory capacity based on the memory capacity of the first GPU and the memory capacity of the second GPU; send the target GPU computing power resources and the target GPU memory capacity to the driver corresponding to the first GPU or the second GPU; and allocate the target GPU computing power resources and the target GPU memory capacity to the first container and the second container through the driver corresponding to the first GPU or the second GPU.

[0177] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: if it is determined that the first GPU and the second GPU are different, sending the computing power resources and the memory capacity of the first GPU to the driver of the first GPU, and sending the computing power resources and the memory capacity of the second GPU to the driver of the second GPU; allocating the computing power resources and the memory capacity of the first GPU to the first container through the driver of the first GPU, and allocating the computing power resources and the memory capacity of the second GPU to the second container through the driver of the second GPU.

[0178] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: performing time-slicing processing on the computing tasks in the first container and the computing tasks in the second container respectively, to obtain multiple first time-slice tasks corresponding to the first container and multiple second time-slice tasks corresponding to the second container; if it is determined that the first GPU and the second GPU are the same, rearranging the multiple first time-slice tasks and the multiple second time-slice tasks based on the computing power resources of the first GPU and the second GPU, to obtain rearranged multiple time-slice tasks; wherein the time-slice ratio of the multiple first time-slice tasks and the multiple second time-slice tasks is the same as the ratio of GPU computing power resources called by each container; sending the rearranged multiple time-slice tasks to the driver of the first GPU or the second GPU, so that the driver of the first GPU or the second GPU executes the rearranged multiple time-slice tasks.

[0179] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: if it is determined that the first GPU and the second GPU are different, sending a plurality of first time slice tasks to the driver of the first GPU so that the driver of the first GPU executes the plurality of first time slice tasks, and sending a plurality of second time slice tasks to the driver of the second GPU so that the driver of the second GPU executes the plurality of second time slice tasks.

[0180] In one embodiment, the GPU resource request further includes indication information for obtaining additional GPU resources; when the computer program is executed by the processor, it also performs the following steps: monitoring the usage information of GPU computing power resources and GPU memory capacity corresponding to the first container through a monitoring thread; when it is determined that the GPU resources required by the first container exceed the GPU resources requested by the first container, and when it is determined that there are idle GPU resources in the GPU, additional GPU resources are allocated to the first container; GPU resources include GPU computing power resources and GPU memory capacity.

[0181] It should be noted that the data involved in this application (including but not limited to GPU resource requests, the first GPU, the first GPU computing power resources, the first GPU video memory capacity, video memory quota, etc.) are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0182] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0183] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0184] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for virtualizing a graphics processing unit (GPU) in a container, characterized in that, The method includes: Upon detecting a GPU resource request submitted through the first container, the monitoring thread corresponding to the first container is activated; the GPU resource request includes the first GPU computing power resources and the first GPU memory capacity of the requested first GPU; Upon detecting a launch operation for an application in the first container, the monitoring thread monitors the call commands from the first container to the first GPU; the call commands are determined by the first container based on the GPU resource requests; the call commands are used to access the first GPU's computing power resources and memory capacity. Based on the invocation command, the computing power resources of the first GPU and the memory capacity of the first GPU are determined; If it is determined that the first GPU memory capacity is less than or equal to the memory quota corresponding to the first container, the first GPU computing power resources and the first GPU memory capacity are sent to the driver corresponding to the first GPU. The first GPU computing power resources and the first GPU memory capacity are allocated to the first container through the driver corresponding to the first GPU.

2. The method according to claim 1, characterized in that, The method further includes: If it is determined that the GPU memory capacity of the first GPU called by the first container is less than or equal to the memory quota corresponding to the first container, and the GPU memory capacity of the second GPU called by the second container is less than or equal to the memory quota corresponding to the second container, then the first GPU corresponding to the first container and the second GPU corresponding to the second container are determined. If the first GPU and the second GPU are determined to be the same, the target GPU computing power resources are determined based on the computing power resources of the first GPU and the computing power resources of the second GPU called by the second container; and the target GPU memory capacity is determined based on the memory capacity of the first GPU and the memory capacity of the second GPU. Send the target GPU computing power resources and the target GPU memory capacity to the driver corresponding to the first GPU or the second GPU; The target GPU computing power resources and target GPU memory capacity are allocated to the first container and the second container through the driver corresponding to the first GPU or the second GPU.

3. The method according to claim 2, characterized in that, The method further includes: If it is determined that the first GPU and the second GPU are different, the computing power resources of the first GPU and the memory capacity of the first GPU are sent to the driver of the first GPU, and the computing power resources of the second GPU and the memory capacity of the second GPU are sent to the driver of the second GPU. The first GPU's computing power resources and memory capacity are allocated to the first container through the first GPU's driver, and the second GPU's computing power resources and memory capacity are allocated to the second container through the second GPU's driver.

4. The method according to claim 3, characterized in that, The method further includes: The computation tasks in the first container and the computation tasks in the second container are processed by time slicing to obtain multiple first time slice tasks corresponding to the first container and multiple second time slice tasks corresponding to the second container. If the first GPU and the second GPU are determined to be the same, the multiple first time slice tasks and the multiple second time slice tasks are rearranged based on the computing power resources of the first GPU and the second GPU to obtain multiple rearranged time slice tasks; wherein, the time slice ratio of the multiple first time slice tasks and the multiple second time slice tasks is the same as the ratio of GPU computing power resources called by each container; The orchestrated multiple time-slice tasks are sent to the driver of the first GPU or the second GPU, so that the driver of the first GPU or the second GPU executes the orchestrated multiple time-slice tasks.

5. The method according to claim 4, characterized in that, The method further includes: If it is determined that the first GPU and the second GPU are different, a plurality of first time slice tasks are sent to the driver of the first GPU so that the driver of the first GPU executes the plurality of first time slice tasks, and a plurality of second time slice tasks are sent to the driver of the second GPU so that the driver of the second GPU executes the plurality of second time slice tasks.

6. The method according to any one of claims 1 to 5, characterized in that, The GPU resource request also includes indication information for obtaining additional GPU resources; the method further includes: The monitoring thread monitors the usage information of the GPU computing power resources and GPU memory capacity corresponding to the first container. If it is determined that the GPU resources required by the first container exceed the GPU resources requested by the first container, and it is determined that there are idle GPU resources in the GPU, additional GPU resources are allocated to the first container; the GPU resources include GPU computing power resources and GPU memory capacity.

7. A virtualization device for a graphics processing unit (GPU) in a container, characterized in that, The device includes: The processing module is used to activate the monitoring thread corresponding to the first container when a GPU resource request submitted through the first container is detected; the GPU resource request includes the first GPU computing power resources and the first GPU memory capacity of the requested first GPU; The monitoring module is used to monitor, through the monitoring thread, the calling commands of the first container to the first GPU when a startup operation for an application in the first container is detected; the calling commands are determined by the first container based on the GPU resource requests; the calling commands are used to call the first GPU computing power resources and the first GPU memory capacity of the first GPU. The determining module is used to determine the computing power resources of the first GPU and the memory capacity of the first GPU based on the calling command; The processing module is used to send the first GPU computing power resources and the first GPU memory capacity to the driver corresponding to the first GPU when it is determined that the first GPU memory capacity is less than or equal to the memory quota corresponding to the first container. The allocation module is used to allocate the computing power resources of the first GPU and the video memory capacity of the first GPU to the first container through the driver corresponding to the first GPU.

8. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 6.

Images