A GPU virtualization method, device, equipment and storage medium

Abstract

The application discloses a GPU virtualization method, device, equipment and storage medium, and relates to the technical field of virtualization. The method comprises the following steps: triggering the loading of a target library file preconfigured in a container group according to a GPU task of an application program running in the container group; intercepting an image processing instruction initiated by the application program in the container group by running the target library file, and judging whether to allocate GPU resources to the container group according to a resource allocation rule; if it is determined to allocate GPU resources to the container group, the image processing instruction is forwarded to a task scheduling component of a host computer; the task scheduling component allocates corresponding virtual GPU resources to the container group according to the GPU task corresponding to the memory requirement amount, and sends a processing result output by the GPU to the container group. The GPU virtualization process is optimized, and the flexibility of GPU virtual resource allocation is improved.

Description

Technical Field

[0001] This invention relates to the field of virtual machine technology, and in particular to a GPU virtualization method, apparatus, device, and storage medium. Background Technology

[0002] GPU (Graphics Processing Unit, used for computer graphics rendering and the training and inference processes of deep learning) virtualization technology is a technique that virtualizes physical GPU resources into multiple virtual GPU resources, allowing multiple virtual machines to share the same one or more GPU processors for graphics computation. However, GPU resource virtualization requires compiling long code to execute the GPU resource allocation process, which reduces the efficiency of GPU virtualization. Summary of the Invention

[0003] In view of this, the purpose of this invention is to provide a GPU virtualization method, apparatus, device, and storage medium that can improve the flexibility of GPU virtual resource allocation. The specific solution is as follows:

[0004] In a first aspect, this application discloses a GPU virtualization method, including:

[0005] Based on the GPU tasks running within the application in the container group, trigger the loading of pre-configured target library files in the container group;

[0006] By running the target library file, the image processing instructions initiated by the application within the container group are intercepted, and it is determined whether to allocate GPU resources to the container group according to the resource allocation rules.

[0007] If it is determined that GPU resources should be allocated to the container group, the image processing instructions are forwarded to the host machine's task scheduling component.

[0008] The task scheduling component allocates corresponding virtual GPU resources to the container group according to the memory requirements of the GPU task, and sends the processing results output by the GPU to the container group.

[0009] Optionally, determining whether to allocate GPU resources to the container group based on resource allocation rules includes:

[0010] Get the total and remaining video memory capacity corresponding to the container group;

[0011] By comparing the remaining video memory capacity with the video memory requirement corresponding to the image processing instructions, it is determined whether the container group needs to allocate more GPU resources.

[0012] If necessary, the container group is determined to meet the GPU resource allocation conditions based on the resource allocation rules corresponding to the container group and the total video memory capacity corresponding to the container group; the resource allocation rules include resource allocation thresholds.

[0013] If the conditions are met, then GPU resources will be allocated to the container group.

[0014] Optionally, the GPU virtualization method further includes:

[0015] The system management commands are intercepted through the target library file; these system management commands are used to query GPU information.

[0016] Replace the host process ID field in the system management command with the container process ID field to obtain a custom query command;

[0017] The GPU process ID corresponding to the container group is retrieved by executing the custom query command.

[0018] Optionally, before allocating corresponding virtual GPU resources to the container group by the task scheduling component according to the video memory requirements corresponding to the GPU task, the method further includes:

[0019] The task scheduling component establishes a connection between the host machine and the target port; the target port is the port corresponding to the container group.

[0020] The host machine uses the connection between the host machine and the target port to listen to the target port and obtain messages sent by the container group;

[0021] The image processing instructions are selected based on the message type of the message.

[0022] Optionally, the step of allocating corresponding virtual GPU resources to the container group through the task scheduling component according to the video memory requirements corresponding to the GPU task includes:

[0023] The target GPU is determined from all GPUs based on the performance requirements of the application;

[0024] Allocate virtual GPU resources corresponding to the target GPU to the container group.

[0025] Optionally, the GPU virtualization method further includes:

[0026] During the execution of the target library file and the task scheduling component, their respective corresponding program error handling is performed;

[0027] The program error handling corresponding to the target library file includes monitoring the communication status between the container group and the host machine, and stopping the execution of the target library file if the communication times out; the program error handling corresponding to the task scheduling component includes generating a running log file according to the log path.

[0028] Optionally, the step of allocating corresponding virtual GPU resources to the container group according to the GPU memory requirements of the GPU task through the task scheduling component, and sending the processing results output by the GPU to the container group, includes:

[0029] The task scheduling component creates a corresponding number of message processing threads based on the number of GPUs contained in the local node; wherein, one message processing thread corresponds to one GPU.

[0030] The task scheduling component listens to the processing results output by each GPU through the message processing thread and forwards the processing results to the corresponding container group.

[0031] Secondly, this application discloses a GPU virtualization device, comprising:

[0032] The target library file loading module is used to trigger the loading of pre-configured target library files in the container group based on the GPU tasks running by the application within the container group;

[0033] The image processing instruction interception module is used to intercept image processing instructions initiated by the application within the container group by running the target library file, and to determine whether to allocate GPU resources to the container group according to the resource allocation rules.

[0034] An image processing instruction forwarding module is used to forward the image processing instructions to the host machine's task scheduling component if it is determined that GPU resources should be allocated to the container group.

[0035] The resource allocation module is used to allocate corresponding virtual GPU resources to the container group according to the video memory requirements of the GPU task through the task scheduling component, and send the processing results output by the GPU to the container group.

[0036] Thirdly, this application discloses an electronic device, including:

[0037] Memory, used to store computer programs;

[0038] A processor for executing the computer program to implement the aforementioned GPU virtualization method.

[0039] Fourthly, this application discloses a computer-readable storage medium for storing a computer program; wherein the computer program implements the aforementioned GPU virtualization method when executed by a processor.

[0040] In this application, a pre-configured target library file in the container group is loaded based on the GPU task running by the application within the container group. The image processing instructions initiated by the application within the container group are intercepted by running the target library file, and a determination is made according to the resource allocation rules whether to allocate GPU resources to the container group. If it is determined that GPU resources should be allocated to the container group, the image processing instructions are forwarded to the host machine's task scheduling component. The task scheduling component allocates corresponding virtual GPU resources to the container group according to the video memory requirements corresponding to the GPU task, and sends the processing results output by the GPU to the container group.

[0041] As can be seen, by passively loading target library files through GPU tasks, image processing instructions initiated by applications within the container group are intercepted. Based on resource allocation rules, and after determining that GPU resources should be allocated to the container group if they meet the rules, the image processing instructions are forwarded to the host machine's task scheduling component, thus enabling control over GPU resources. The host machine then uniformly receives the image processing tasks corresponding to each container group and allocates the necessary virtual GPU resources to each GPU task's corresponding container group. Each application can interact with the GPU without interfering with other applications, ensuring data security and privacy. This optimizes the GPU virtualization process and improves the flexibility of GPU virtual resource allocation. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0043] Figure 1 A flowchart of a GPU virtualization method provided in this application;

[0044] Figure 2 This application provides a specific flowchart of the target library file execution process;

[0045] Figure 3 This application provides a schematic diagram of a specific GPU virtualization structure;

[0046] Figure 4 This application provides a schematic diagram of a GPU virtualization device structure.

[0047] Figure 5 This application provides a structural diagram of an electronic device. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] In existing technologies, GPU resource virtualization requires compiling long code to execute the GPU resource allocation process, which reduces the efficiency of GPU virtualization. To overcome the above technical problems, this application proposes a GPU virtualization method that can optimize the GPU virtualization process and improve the flexibility of GPU virtual resource allocation.

[0050] This application discloses a GPU virtualization method. See also Figure 1 As shown, the method may include the following steps:

[0051] Step S11: Based on the GPU tasks running in the application within the container group, trigger the loading of pre-configured target library files in the container group.

[0052] In this embodiment, each container group (i.e., POD, the smallest scheduling unit in Kubernetes) is pre-configured with a target library file. This target library file runs within the user's development environment (i.e., within a container or POD). The target library file is called passively, meaning that it is triggered when the user runs a GPU task. The target library file is specifically a .so file, which can be used for instruction hijacking and redefining function calls.

[0053] Step S12: Intercept the image processing instructions initiated by the application in the container group by running the target library file, and determine whether to allocate GPU resources to the container group according to the resource allocation rules.

[0054] After calling the target library file, the system intercepts image processing instructions initiated by the application within the container group containing the library file by running that file. These instructions can be training instructions issued by the application's training framework, i.e., instructions requiring GPU processing. These instructions contain various information related to the GPU task, including but not limited to the task type and the data to be processed. Then, based on resource allocation rules, it determines whether GPU resources should be allocated to the container group. If so, the image processing instructions are forwarded to the host machine's task scheduling component. It's understood that each application's call to the GPU is not initiated directly; instead, the target library file in each container group first intercepts and judges the request. If it conforms to the resource allocation rules, it is forwarded to the host. These rules include a resource allocation limit, where GPU memory refers to the GPU's internal memory. The resource allocation rules can control the upper limit of allocated GPU memory and various resources, such as the GPU process duration corresponding to a GPU task, thereby increasing control over resource allocation.

[0055] In this embodiment, determining whether to allocate GPU resources to the container group based on resource allocation rules may include: obtaining the total and remaining video memory capacity corresponding to the container group; comparing the remaining video memory capacity with the video memory requirement corresponding to the image processing instructions to determine whether the container group needs to be allocated more GPU resources; if so, determining whether the container group meets the GPU resource allocation conditions based on the resource allocation rules corresponding to the container group and the total video memory capacity corresponding to the container group; the resource allocation rules include a resource allocation threshold; if met, it is determined that GPU resources are allocated to the container group. Specifically, by running the target library file, the total and remaining video memory capacity corresponding to the container group are obtained, i.e., the total video memory historically allocated to the container group, and the remaining video memory capacity after deducting the currently used video memory capacity of the container group. Combined with the video memory requirement of the current task, if the video memory requirement is greater than the remaining video memory capacity, then GPU resources need to be allocated; if the video memory requirement is less than the remaining video memory capacity, then no allocation is needed, i.e., no forwarding of image processing instructions is required. Next, it's necessary to determine if the GPU resources in this node still meet the allocation requirements, i.e., whether there are enough GPU resources available for the container group. If there are enough resources, proceed to the next step; otherwise, do not forward image processing instructions. Furthermore, it's also necessary to determine if the container group meets the GPU resource allocation conditions based on the resource allocation rules and the total video memory capacity corresponding to the container group. The resource allocation rules include resource allocation thresholds, which determine whether to continue allocating resources based on the current resource usage of the container group and the resource usage limit for that container group. For example, if the video memory limit for the container group is pre-configured to 5GB, and 5GB of resources has already been allocated, then no further resources will be allocated when the container group requests resources again. This ensures that all applications within different container groups can use video memory resources.

[0056] For example Figure 2 As shown, the main functions of the target library file (libcuda_nvml_partition.so) include POD memory control and POD process display. Specifically, in this embodiment, the step of determining whether to allocate GPU resources to the container group according to resource allocation rules includes: intercepting the dynamic library loading function through the target library file and redefining it to obtain a custom dynamic library loading function; calling the custom dynamic library loading function when the user triggers the first function; redefining the first function using the custom dynamic library loading function to obtain a custom first function; and executing the custom first function to perform the operation of determining whether to allocate GPU resources to the container group according to resource allocation rules. It is understandable that this embodiment improves upon the original GPU virtualization solution. The original solution could not control video memory, so by configuring the target library file, upon initial runtime, the original dynamic library loading function (dlsym) is intercepted and redefined as a custom dynamic library loading function, so that the custom dynamic library loading function is called when the user triggers the first function for the first time. It is also understandable that the system has a custom dynamic library loading function, but due to version limitations, in order to ensure that the dynamic library loading function can be triggered and called in order to define business functions, a new custom dynamic library loading function is required.

[0057] The first function is redefined using a custom dynamic library loading function, resulting in a custom first function. Executing this custom first function performs the operation of determining whether to allocate GPU resources to the container group based on resource allocation rules. This first function is a CUDA function; after redefinition, it implements container group memory control. Specifically, it adds container group memory control functionality by replacing the original framework's Application Programming Interface (API) with a custom function in this embodiment. For implicitly allocated memory sizes that cannot be obtained (such as CUDA context), the memory size of the corresponding GPU process is obtained by calling the nvml function. This memory is managed by the driver and is highly accurate. Simultaneously, based on the container GPU process information in shared memory, the memory size used by all GPU processes within the container is obtained. Combined with the container quota and the size to be allocated, memory is precisely controlled.

[0058] Specifically, taking GPU virtualization under the NVIDIA architecture as an example, the code that determines whether to allocate GPU resources to the container group based on resource allocation rules is as follows:

[0059]

[0060]

[0061] In this embodiment, after the task scheduling component allocates corresponding virtual GPU resources to the container group according to the GPU task's memory requirements, it may further include: intercepting system management commands through the target library file; the system management commands are used to query GPU information; replacing the host process ID field in the system management commands with the container process ID field to obtain a custom query command; and querying the GPU process ID (PID, processID) corresponding to the container group by executing the custom query command. It is understood that the original system management commands are invalid within the container group, meaning they cannot be executed within the POD and can only be executed on the host. Since there are many PODs and many users on the host, if the original system management commands are executed to query processes in the original way, the host PID will be obtained, which would leak data from other PODs. By redefining the query command, executing the query command only tells the user the process ID related to the POD, i.e., the POD PID. It is understood that after a function redefinition, subsequent executions will follow the redefined function.

[0062] In this embodiment, before intercepting system management commands through the target library file, the method further includes: when the user triggers the second function, calling the custom dynamic library loading function, redefining the second function using the custom dynamic library loading function to obtain a custom second function; and executing the custom second function to perform the interception of system management commands through the target library file and subsequent operations. Therefore, by intercepting system management commands and replacing them with process query commands that can be used for containers, the target library file enables the display of GPU processes within the container.

[0063] Specifically, taking GPU virtualization under the NVIDIA architecture as an example, the second function mentioned above is the nvai function, and the system management command mentioned above is the nvidia-smi command. The code for the container group process ID display process is shown below:

[0064]

[0065]

[0066] Step S13: If it is determined that GPU resources should be allocated to the container group, the image processing instructions are forwarded to the host machine's task scheduling component.

[0067] If the conditions are met and GPU resources can be allocated to the container group, the aforementioned image processing instructions are forwarded to the host machine's task scheduling component. If the resource allocation conditions are not met, the image processing instructions are not processed.

[0068] Step S14: The task scheduling component allocates corresponding virtual GPU resources to the container group according to the memory requirements of the GPU task, and sends the processing results output by the GPU to the container group.

[0069] In this embodiment, the task scheduling component on the host machine allocates GPU resources according to the video memory requirements of the GPU tasks and allocates corresponding virtual GPU resources to the container group. At the same time, it sends the corresponding GPU process ID on the host machine to the container group after allocation and sends the processing results output by the GPU to the container group.

[0070] In this embodiment, before allocating corresponding virtual GPU resources to the container group according to the GPU memory requirements of the GPU task through the task scheduling component, the process may further include: establishing a connection between the host machine and the target port through the task scheduling component; the target port being the port corresponding to the container group; listening to the target port using the connection between the host machine and the target port to obtain messages sent by the container group; and filtering the image processing instructions according to the message type of the messages. Specifically, the host machine and the container communicate using the UDP (User Datagram Protocol) protocol. Specifically, the target port of the container component is read from the configuration file (port.conf), a UDP connection is created between the host machine and the target port, and the host machine listens for messages. It can be understood that all messages sent by the container group will be sent to the host machine through the target port, and the sent messages contain not only image processing instructions but also other instructions. Therefore, it is necessary to filter the image processing instructions according to the message type. For example, if the message type of the image processing instruction is 10, then only messages of message type 10 can be processed, and other types are ignored.

[0071] In this embodiment, the step of allocating corresponding virtual GPU resources to the container group by the task scheduling component based on the video memory requirements of the GPU task may include: determining the target GPU from all GPUs according to the performance requirements of the application; and allocating the virtual GPU resources corresponding to the target GPU to the container group. This allows for different GPU resource allocations to be provided to different applications, thereby meeting the performance requirements of different applications.

[0072] Furthermore, in this embodiment, the application runs within a container within a container group. Image processing instructions are forwarded to the host via API, where resources are allocated centrally. Each application can interact with the GPU without interfering with other applications, ensuring data security and privacy. Each application also has its own virtual GPU environment, so even if one application malfunctions or crashes, it will not affect the normal operation of other applications. Simultaneously, GPU virtualization technology enables operation on different operating systems and platforms, thereby improving compatibility and portability.

[0073] In this embodiment, the step of allocating corresponding virtual GPU resources to the container group based on the GPU task's memory requirements using the task scheduling component, and sending the GPU's processing results to the container group, can include: creating a corresponding number of message processing threads based on the number of GPUs in the local node using the task scheduling component; wherein one message processing thread corresponds to one GPU; the task scheduling component listens to the processing results output by each GPU through the message processing threads, and forwards the processing results to the corresponding container group. That is, corresponding message processing threads are created based on the number of GPUs in the node, each thread only processes messages for its corresponding GPU, and after thread creation, the thread ID is sent to the corresponding container group. This thread is used to listen to the GPU's processing results. In this embodiment, the task scheduling component (host_proc_ctled) runs on the host as a daemon process, with a mechanism for automatic startup upon system boot or restart; however, actively killing the process will not result in automatic restart.

[0074] In this embodiment, the GPU virtualization method may further include: executing corresponding program error handling during the operation of the target library file and the task scheduling component; the program error handling corresponding to the task scheduling component includes generating a runtime log file based on the log path. It is understood that program error handling refers to the process of discovering and processing errors during program execution. Error handling is an unavoidable part of program development. The purpose of error handling is to identify and correct errors in the program to ensure that the program runs as expected. The task scheduling component's error log handling will determine whether a preset log path exists. If it exists, a directory for GPU virtualization will be created under that log path, and a log file will be generated based on the current time; if it does not exist, no log will be generated. In addition, it is necessary to determine whether a configuration file is missing. This configuration file contains the target port corresponding to the container group; if it is missing, a prompt to set the corresponding file is required. The error handling for the target library file includes monitoring the communication status between the container group and the host machine. If the communication times out, the target library file will stop running. For example, if the communication timeout for the target library file to obtain the GPU process ID sent by the host is set to 90 seconds, the target library file will exit directly if the timeout occurs or the process ID is parsed incorrectly. For details, please refer to the exit stack information.

[0075] For example Figure 3The diagram illustrates a specific GPU virtualization system. When a user launches a deep learning task within a container, the task can be run using a training framework (such as TensorFlow or PyTorch). The training framework issues various instructions to execute the training process. However, these instructions are intercepted by a target library file before being sent to the system's underlying layer. This library file can observe and modify the instructions issued by the framework. The target library file includes a function called a communicator, whose main function is to receive instructions from the training framework, parse the information within them, and specifically extract various task-related information, such as the task type and the data to be processed. The target library file parses the information in the instructions and sends this information to the host machine's task scheduling component (host_proc_ctled). This component acts as the task scheduler, forwarding the corresponding computational requests to appropriate hardware resources based on the received information, i.e., sending these computational requests to the GPU hardware on the physical host. Specifically, the communicator in the task scheduling component is used for instruction forwarding with the POD, and the PID map manager module is used for resource scheduling. Once the GPU begins processing these computational tasks, the task scheduling component closely monitors their execution. When a task completes, the task scheduling component retrieves the results from the GPU and returns them to the POD. The POD can then determine its next steps based on these results. This method enables the use of virtualized GPUs, allowing users to instruct the training framework on how to use the GPU for computation with simple commands, without needing to concern themselves with the specific hardware details.

[0076] As can be seen from the above, in this embodiment, the loading of a pre-configured target library file in the container group is triggered based on the GPU task running by the application within the container group. The image processing instructions initiated by the application within the container group are intercepted by running the target library file, and a determination is made according to resource allocation rules to decide whether to allocate GPU resources to the container group. If it is determined that GPU resources should be allocated to the container group, the image processing instructions are forwarded to the host machine's task scheduling component. The task scheduling component allocates corresponding virtual GPU resources to the container group according to the video memory requirements of the GPU task and sends the processing results output by the GPU to the container group. Therefore, by passively loading the target library file through the GPU task, intercepting the image processing instructions initiated by the application within the container group, and judging according to resource allocation rules, if it is determined that GPU resources should be allocated to the container group according to the resource allocation rules, the image processing instructions are forwarded to the host machine's task scheduling component, thus enabling control over GPU resources. The host machine then uniformly receives the image processing tasks corresponding to each container group and allocates the required virtual GPU resources to the container group corresponding to each GPU task. Each application can interact with the GPU without interfering with the existence of other applications, ensuring data security and privacy. The GPU virtualization process has been optimized to improve the flexibility of GPU virtual resource allocation.

[0077] Accordingly, this application also discloses a GPU virtualization device, see [link to relevant documentation]. Figure 4 As shown, the device includes:

[0078] The target library file loading module 11 is used to trigger the loading of pre-configured target library files in the container group based on the GPU tasks run by the application within the container group.

[0079] The image processing instruction interception module 12 is used to intercept the image processing instructions initiated by the application in the container group by running the target library file, and to determine whether to allocate GPU resources to the container group according to the resource allocation rules.

[0080] The image processing instruction forwarding module 13 is used to forward the image processing instruction to the host machine's task scheduling component if it is determined that GPU resources will be allocated to the container group.

[0081] The resource allocation module 14 is used to allocate corresponding virtual GPU resources to the container group according to the video memory requirements of the GPU task through the task scheduling component, and send the processing results output by the GPU to the container group.

[0082] As can be seen from the above, in this embodiment, the loading of a pre-configured target library file in the container group is triggered based on the GPU task running by the application within the container group; the image processing instructions initiated by the application within the container group are intercepted by running the target library file, and it is determined whether to allocate GPU resources to the container group according to the resource allocation rules. If it is determined that GPU resources should be allocated to the container group, the image processing instructions are forwarded to the host machine's task scheduling component; the task scheduling component allocates corresponding virtual GPU resources to the container group according to the video memory requirements corresponding to the GPU task, and sends the processing results output by the GPU to the container group. As can be seen, by passively loading target library files through GPU tasks, intercepting image processing instructions initiated by applications within the container group, and judging according to resource allocation rules, if it is determined that GPU resources should be allocated to the container group in accordance with the resource allocation rules, the image processing instructions are forwarded to the host machine's task scheduling component. This enables control over GPU resources. The host machine then uniformly receives the image processing tasks corresponding to each container group and allocates the required virtual GPU resources to the container group corresponding to each GPU task. Each application can interact with the GPU without interfering with the existence of other applications, ensuring data security and privacy, optimizing the GPU virtualization process, and improving the flexibility of GPU virtual resource allocation.

[0083] In some specific embodiments, the image processing instruction interception module 12 may specifically include:

[0084] The container group video memory information acquisition unit is used to acquire the total video memory capacity and remaining video memory capacity corresponding to the container group;

[0085] The first judgment unit is used to determine whether the container group needs to allocate more GPU resources by comparing the remaining video memory capacity with the video memory requirement corresponding to the image processing instruction.

[0086] The second judgment unit is used to determine, if necessary, whether the container group meets the GPU resource allocation conditions based on the resource allocation rules corresponding to the container group and the total video memory capacity corresponding to the container group; the resource allocation rules include resource allocation thresholds.

[0087] The resource allocation determination unit is used to determine, if satisfied, to allocate GPU resources to the container group.

[0088] In some specific embodiments, the image processing instruction interception module 12 may specifically include:

[0089] The system management command interception unit is used to intercept system management commands through the target library file; the system management commands are used to query GPU information.

[0090] The replacement unit is used to replace the host process ID field in the system management command with the container process ID field to obtain a custom query command;

[0091] The process query unit is used to query the GPU process ID corresponding to the container group by executing the custom query command.

[0092] In some specific embodiments, the GPU virtualization device may specifically include:

[0093] A connection establishment unit is used to establish a connection between the host machine and a target port through the task scheduling component before allocating corresponding virtual GPU resources to the container group according to the video memory requirements of the GPU task through the task scheduling component; the target port is the port corresponding to the container group.

[0094] The port listening unit is used to listen to the target port by utilizing the connection between the host machine and the target port, and to obtain the messages sent by the container group;

[0095] An image processing instruction filtering unit is used to filter out the image processing instructions according to the message type of the message.

[0096] In some specific embodiments, the resource allocation module 14 may specifically include:

[0097] The target GPU determination unit is used to determine the target GPU from all GPUs based on the performance requirements of the application.

[0098] The resource allocation unit is used to allocate virtual GPU resources corresponding to the target GPU to the container group.

[0099] In some specific embodiments, the GPU virtualization device may specifically include:

[0100] The error handling unit is used to execute the corresponding program error handling during the operation of the target library file and the task scheduling component.

[0101] The program error handling corresponding to the target library file includes monitoring the communication status between the container group and the host machine, and stopping the execution of the target library file if the communication times out; the program error handling corresponding to the task scheduling component includes generating a running log file according to the log path.

[0102] In some specific embodiments, the resource allocation module 14 may specifically include:

[0103] The message processing thread creation unit is used to create a corresponding number of message processing threads based on the number of GPUs contained in the local node through the task scheduling component; wherein, one message processing thread corresponds to one GPU.

[0104] The processing result monitoring unit is used by the task scheduling component to monitor the processing results output by each GPU through the message processing thread, and forward the processing results to the corresponding container group.

[0105] Furthermore, this application also discloses an electronic device, see [link to relevant documentation]. Figure 5 As shown, the content in the figure should not be considered as any limitation on the scope of use of this application.

[0106] Figure 5 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the GPU virtualization method disclosed in any of the foregoing embodiments.

[0107] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0108] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon include operating system 221, computer program 222 and data 223 including target library files, etc. The storage method can be temporary storage or permanent storage.

[0109] The operating system 221 manages and controls the various hardware devices on the electronic device 20 and the computer program 222 to enable the processor 21 to perform calculations and processing on the massive amounts of data 223 in the memory 22. The operating system 221 can be Windows Server, Netware, Unix, Linux, etc. In addition to including computer programs capable of performing the GPU virtualization method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0110] Furthermore, this application also discloses a computer storage medium storing computer-executable instructions. When the computer-executable instructions are loaded and executed by a processor, they implement the GPU virtualization method steps disclosed in any of the foregoing embodiments.

[0111] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0112] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0113] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0114] The foregoing has provided a detailed description of a GPU virtualization method, apparatus, device, and storage medium provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A GPU virtualization method, characterized in that, include: Based on the GPU tasks running within the application in the container group, trigger the loading of pre-configured target library files in the container group; By running the target library file, the image processing instructions initiated by the application within the container group are intercepted, and it is determined whether to allocate GPU resources to the container group according to the resource allocation rules. The resource allocation rules include: determining whether the container group needs to allocate more GPU resources based on the remaining video memory capacity of the container group and the video memory requirement corresponding to the image processing instruction; and determining whether the container group meets the GPU resource allocation conditions based on the total video memory capacity corresponding to the container group and the resource allocation threshold. If it is determined that GPU resources should be allocated to the container group, the image processing instructions are forwarded to the host machine's task scheduling component. The task scheduling component allocates corresponding virtual GPU resources to the container group according to the memory requirements of the GPU task, and sends the processing results output by the GPU to the container group.

2. The GPU virtualization method according to claim 1, characterized in that, The step of determining whether to allocate GPU resources to the container group according to the resource allocation rules includes: Get the total and remaining video memory capacity corresponding to the container group; By comparing the remaining video memory capacity with the video memory requirement corresponding to the image processing instructions, it is determined whether the container group needs to allocate more GPU resources. If necessary, the container group is determined to meet the GPU resource allocation conditions based on the resource allocation rules corresponding to the container group and the total video memory capacity corresponding to the container group; the resource allocation rules include resource allocation thresholds. If the conditions are met, then GPU resources will be allocated to the container group.

3. The GPU virtualization method according to claim 1, characterized in that, After the task scheduling component allocates corresponding virtual GPU resources to the container group according to the memory requirements of the GPU task, the method further includes: The system management commands are intercepted through the target library file; these system management commands are used to query GPU information. Replace the host process ID field in the system management command with the container process ID field to obtain a custom query command; The GPU process ID corresponding to the container group is retrieved by executing the custom query command.

4. The GPU virtualization method according to claim 1, characterized in that, Before the task scheduling component allocates corresponding virtual GPU resources to the container group according to the memory requirements of the GPU task, the method further includes: The task scheduling component establishes a connection between the host machine and the target port; the target port is the port corresponding to the container group. The host machine uses the connection between the host machine and the target port to listen to the target port and obtain messages sent by the container group; The image processing instructions are selected based on the message type of the message.

5. The GPU virtualization method according to claim 1, characterized in that, The step of allocating corresponding virtual GPU resources to the container group through the task scheduling component based on the GPU memory requirements corresponding to the GPU task includes: The target GPU is determined from all GPUs based on the performance requirements of the application; Allocate virtual GPU resources corresponding to the target GPU to the container group.

6. The GPU virtualization method according to claim 1, characterized in that, Also includes: During the execution of the target library file and the task scheduling component, their respective corresponding program error handling is performed; The program error handling corresponding to the target library file includes monitoring the communication status between the container group and the host machine, and stopping the execution of the target library file if the communication times out; the program error handling corresponding to the task scheduling component includes generating a running log file according to the log path.

7. The GPU virtualization method according to any one of claims 1 to 6, characterized in that, The step of allocating corresponding virtual GPU resources to the container group according to the GPU memory requirements of the GPU task through the task scheduling component, and sending the processing results output by the GPU to the container group, includes: The task scheduling component creates a corresponding number of message processing threads based on the number of GPUs contained in the local node; wherein, one message processing thread corresponds to one GPU. The task scheduling component listens to the processing results output by each GPU through the message processing thread and forwards the processing results to the corresponding container group.

8. A GPU virtualization device, characterized in that, include: The target library file loading module is used to trigger the loading of pre-configured target library files in the container group based on the GPU tasks running by the application within the container group; The image processing instruction interception module is used to intercept image processing instructions initiated by the application within the container group by running the target library file, and to determine whether to allocate GPU resources to the container group according to the resource allocation rules. The resource allocation rules include: determining whether the container group needs to allocate more GPU resources based on the remaining video memory capacity of the container group and the video memory requirement corresponding to the image processing instruction; and determining whether the container group meets the GPU resource allocation conditions based on the total video memory capacity corresponding to the container group and the resource allocation threshold. An image processing instruction forwarding module is used to forward the image processing instructions to the host machine's task scheduling component if it is determined that GPU resources should be allocated to the container group. The resource allocation module is used to allocate corresponding virtual GPU resources to the container group according to the video memory requirements of the GPU task through the task scheduling component, and send the processing results output by the GPU to the container group.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the GPU virtualization method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store computer programs; wherein the computer programs, when executed by a processor, implement the GPU virtualization method as described in any one of claims 1 to 7.

Images