GPU virtualized video memory management method, device, storage medium, and program product

Abstract

The present disclosure relates to the field of GPU technology, and in particular to a method, device, storage medium and program product for GPU virtualized video memory management. The method comprises: the virtual machine transfers the page table root address corresponding to the target application running in the virtual machine to the host machine; the virtual machine intercepts the call of the non-memory operation class API in response to the target application calling the non-memory operation class API and forwards it to the host machine; in the host machine, the driver of the non-memory operation class API accesses the video memory area corresponding to the target application based on the page table root address, executes the instructions of the non-memory operation class API, and returns the processing results of the instructions of the non-memory operation class API to the virtual machine. The present disclosure enables the back-end driver in the host machine to directly access the video memory resources allocated to the virtual machine, thereby improving resource utilization and task execution efficiency.

Description

Technical Field

[0001] The present disclosure relates to the field of GPU technology, and in particular to a method for managing video memory for GPU virtualization, a device for managing video memory for GPU virtualization, a non-volatile computer-readable storage medium, and a computer program product. Background Art

[0002] Graphics Processing Unit (GPU) virtualization technology plays a crucial role in modern computing environments, particularly in cloud computing and virtual desktop infrastructure (VDI). The rise of artificial intelligence (AI) and large-scale model applications has placed higher demands on efficient GPU resource utilization and performance optimization. However, related technologies still face challenges in hybrid virtualization scenarios, such as insufficient video memory allocation and performance loss. Summary of the Invention

[0003] In view of this, the present disclosure provides a technical solution for video memory management of GPU virtualization.

[0004] According to one aspect of the present disclosure, a method for managing video memory for GPU virtualization is provided, comprising:

[0005] The virtual machine transmits a page table root address corresponding to a target application running in the virtual machine to a host machine; wherein the virtual machine is configured with a virtual GPU, the virtual GPU is allocated a dedicated video memory area, and the video memory area corresponding to the target application belongs to the dedicated video memory area;

[0006] In response to the target application calling a non-memory operation class API, the virtual machine intercepts the call of the non-memory operation class API and forwards the call to the host machine;

[0007] In the host machine, the graphics memory area corresponding to the target application is accessed based on the page table root address through the driver of the non-memory operation class API, the instructions of the non-memory operation class API are executed, and the processing results of the instructions of the non-memory operation class API are returned to the virtual machine.

[0008] In one possible implementation,

[0009] The method further includes: the virtual machine sending an identifier of the virtual machine and a process identifier of the target application to the host machine;

[0010] The method of accessing the video memory area corresponding to the target application based on the page table root address through the driver of the non-memory operation class API in the host machine and executing the instructions of the non-memory operation class API includes: accessing the video memory area corresponding to the target application based on the identifier of the virtual machine, the process identifier of the target application and the page table root address through the driver of the non-memory operation class API in the host machine and executing the instructions of the non-memory operation class API.

[0011] In a possible implementation, before transferring the page table root address corresponding to the target application running in the virtual machine to the host machine, the method further includes:

[0012] Loading a GPU driver in the host machine and creating multiple virtual GPUs, and allocating dedicated video memory areas to each of the multiple virtual GPUs;

[0013] The host machine transparently transmits the virtual GPU to the virtual machine.

[0014] In a possible implementation, the sum of the capacities of the dedicated video memory areas of the multiple virtual GPUs is equal to the total video memory capacity of the physical GPU.

[0015] In a possible implementation, in response to the target application calling a non-memory operation API, the virtual machine intercepts the call of the non-memory operation API and forwards the call to the host machine, including:

[0016] In response to detecting, through the virtualization layer, that the target application program calls a non-memory operation class API, the virtual machine intercepts the call of the non-memory operation class API through the virtualization layer and forwards the call to the host machine.

[0017] In one possible implementation,

[0018] The method further includes: the virtual machine processing instructions of the memory operation class API in response to the target application calling the memory operation class API, and recording an updated page table root address corresponding to the target application;

[0019] The virtual machine transmits the page table root address corresponding to the target application running in the virtual machine to the host machine, including: the virtual machine transmits the updated page table root address corresponding to the target application to the host machine in response to the page table root address corresponding to the target application being updated.

[0020] In one possible implementation, the memory operation API includes:

[0021] API for memory allocation;

[0022] and / or,

[0023] API for memory release.

[0024] In one possible implementation, the target application includes a computationally intensive application.

[0025] In one possible implementation, the computationally intensive application includes an application based on an artificial intelligence model.

[0026] In a possible implementation, the method further includes:

[0027] For the non-computation-intensive application program running in the virtual machine, instructions of the graphics API called by the non-computation-intensive application program are executed through the virtual GPU driver in the virtual machine.

[0028] In a possible implementation, the non-computation-intensive application includes a graphics rendering application.

[0029] In one possible implementation, the method is applied to a scenario where API forwarding and GPU sharding virtualization are mixed.

[0030] According to another aspect of the present disclosure, a graphics memory management device for GPU virtualization is provided, comprising:

[0031] a transfer module, configured for the virtual machine to transfer a page table root address corresponding to a target application running in the virtual machine to a host machine; wherein the virtual machine is configured with a virtual GPU, the virtual GPU is allocated a dedicated video memory area, and the video memory area corresponding to the target application belongs to the dedicated video memory area;

[0032] a forwarding module, configured for the virtual machine to intercept the call of the non-memory operation class API in response to the target application calling the non-memory operation class API and forward the call to the host machine;

[0033] The first processing module is used to access the video memory area corresponding to the target application based on the page table root address through the driver of the non-memory operation class API in the host machine, execute the instructions of the non-memory operation class API, and return the processing results of the instructions of the non-memory operation class API to the virtual machine.

[0034] In one possible implementation,

[0035] The apparatus further includes: a sending module, configured for the virtual machine to send an identifier of the virtual machine and a process identifier of the target application to the host machine;

[0036] The first processing module is used to: in the host machine, access the video memory area corresponding to the target application based on the identifier of the virtual machine, the process identifier of the target application and the page table root address through the driver of the non-memory operation class API, and execute the instructions of the non-memory operation class API.

[0037] In a possible implementation, the apparatus further includes:

[0038] A creation and allocation module, configured to load a GPU driver in the host machine and create multiple virtual GPUs, and allocate dedicated video memory areas to each of the multiple virtual GPUs;

[0039] The transparent transmission module is used for the host machine to transparently transmit the virtual GPU to the virtual machine.

[0040] In a possible implementation, the sum of the capacities of the dedicated video memory areas of the multiple virtual GPUs is equal to the total video memory capacity of the physical GPU.

[0041] In a possible implementation, the forwarding module is configured to:

[0042] In response to detecting, through the virtualization layer, that the target application program calls a non-memory operation class API, the virtual machine intercepts the call of the non-memory operation class API through the virtualization layer and forwards the call to the host machine.

[0043] In one possible implementation,

[0044] The apparatus further includes: a second processing module configured to, in response to the target application calling a memory operation class API, process instructions of the memory operation class API in the virtual machine and record an updated page table root address corresponding to the target application;

[0045] The transfer module is configured to: in response to an update of a page table root address corresponding to the target application, the virtual machine transfers the updated page table root address corresponding to the target application to the host machine.

[0046] In one possible implementation, the memory operation API includes:

[0047] API for memory allocation;

[0048] and / or,

[0049] API for memory release.

[0050] In one possible implementation, the target application includes a computationally intensive application.

[0051] In one possible implementation, the computationally intensive application includes an application based on an artificial intelligence model.

[0052] In a possible implementation, the apparatus further includes:

[0053] An execution module is configured to execute, for the non-computation-intensive application running in the virtual machine, instructions of a graphics API called by the non-computation-intensive application through a virtual GPU driver in the virtual machine.

[0054] In a possible implementation, the non-computation-intensive application includes a graphics rendering application.

[0055] In a possible implementation, the device is applied to a scenario where API forwarding and GPU sharding virtualization are mixed.

[0056] According to another aspect of the present disclosure, a graphics memory management device for GPU virtualization is provided, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above method.

[0057] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented.

[0058] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, or a non-volatile computer-readable storage medium carrying the computer program, wherein the computer program implements the steps of the above method when executed by a processor.

[0059] In an embodiment of the present disclosure, a page table root address corresponding to a target application running in the virtual machine is passed to a host machine through a virtual machine, wherein the virtual machine is configured with a virtual GPU, and the virtual GPU is allocated a dedicated video memory area. The video memory area corresponding to the target application belongs to the dedicated video memory area. In response to the target application calling a non-memory operation class API, the virtual machine intercepts the call of the non-memory operation class API and forwards the call to the host machine. In the host machine, a driver of the non-memory operation class API accesses the video memory area corresponding to the target application based on the page table root address, executes the instructions of the non-memory operation class API, and returns the processing results of the instructions of the non-memory operation class API to the virtual machine, thereby enabling a back-end driver (such as a Vulkan driver) in the host machine to directly access the video memory resources allocated by the virtual machine. That is, the back-end driver in the host machine can share and use the video memory resources allocated by the virtual machine, avoiding the performance bottleneck of being forced to use system memory due to insufficient video memory of the host machine. The disclosed embodiments not only improve the execution efficiency of API forwarding, especially reduce data transmission delay when processing compute-intensive applications (such as applications based on artificial intelligence models), but also enhance the isolation of video memory between different virtual machines, thereby optimizing GPU resource utilization in hybrid virtualization scenarios.

[0060] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0062] Figure 1 A flowchart of a method for managing video memory for GPU virtualization provided by an embodiment of the present disclosure is shown.

[0063] Figure 2 A schematic diagram of a method for managing video memory for GPU virtualization provided by an embodiment of the present disclosure is shown.

[0064] Figure 3 A block diagram of a graphics memory management device for GPU virtualization provided by an embodiment of the present disclosure is shown.

[0065] Figure 4 The present invention is a block diagram showing a device for managing video memory for GPU virtualization according to an exemplary embodiment. DETAILED DESCRIPTION

[0066] Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.

[0067] As used herein, the terms "comprises," "comprising," "having," or variations thereof are open ended and include one or more stated features, integers, elements, steps, parts, or functions, but do not preclude the presence or addition of one or more other features, integers, elements, steps, parts, functions, or groups thereof.

[0068] When an element is referred to as being "connected," "coupled," "responsive" or variations thereof to another element, it can be directly connected, coupled or responsive to the other element or intervening elements may be present.

[0069] Although the terms first, second, third, etc. may be used herein to describe various elements / operations, these elements / operations should not be limited by these terms. These terms are only used to distinguish one element / operation from another element / operation. Therefore, without departing from the teachings of the present invention, the first element / operation in some embodiments may be referred to as the second element / operation in other embodiments.

[0070] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0071] In addition, numerous specific details are provided in the following detailed description to better illustrate the present disclosure. Those skilled in the art will appreciate that the present disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art are not described in detail in order to highlight the main points of the present disclosure.

[0072] Currently, GPU virtualization technologies mainly include the following solutions:

[0073] 1. API forwarding (Application Programming Interface Forwarding)

[0074] API forwarding technology forwards calls to graphics APIs (Application Programming Interfaces) or compute APIs (such as OpenGL, Vulkan, Direct3D, and CUDA) in a guest VM to the host operating system (Host OS) for execution. The host interprets the API call, completes the operation through the GPU driver, and returns the result to the VM.

[0075] The advantage of API forwarding is that it supports multiple virtual machines sharing a physical GPU without requiring GPU hardware support for virtualization. However, its performance depends on the efficiency of API forwarding, especially when processing video memory resources, which may be a bottleneck.

[0076] 2. GPU passthrough

[0077] This is a simple and mature technology that uses PCIe passthrough (Peripheral Component Interconnect Express Passthrough) to directly assign a physical GPU to a single virtual machine. This allows the virtual machine to achieve near-native GPU performance, but the downside is that GPU resources cannot be shared across multiple virtual machines, resulting in low resource utilization.

[0078] 3. GPU sharding virtualization

[0079] This technology, based on a mediated passthrough mechanism, partitions a physical GPU into multiple virtual GPUs (vGPUs), assigning one vGPU to each virtual machine. While the vGPU appears to the virtual machine as an independent device, it actually shares some of the physical GPU's resources through a mediation layer. This solution is suitable for GPUs that don't support hardware virtualization, such as SR-IOV (Single Root Input / Output Virtualization), but it does have limitations in video memory allocation.

[0080] 4. GPU hardware virtualization based on SR-IOV

[0081] SR-IOV is a hardware-level virtualization technology supported by the PCIe (Peripheral Component Interconnect Express) standard. It can virtualize a single PCIe physical function (PF) into multiple independent virtual functions (VFs) at the hardware level.

[0082] When applied to GPUs, this technology can divide a physical GPU into multiple virtual GPUs. Each virtual machine can "pass through" a virtual GPU, that is, each virtual GPU can be directly assigned to a different virtual machine for use, allowing the virtual machine to obtain a nearly native GPU performance experience.

[0083] In practical applications, hybrid virtualization solutions combining API forwarding and GPU sharding are becoming mainstream. API forwarding is primarily used for AI (artificial intelligence) computing scenarios within Windows virtual machines (Windows guests). For example, when a Vulkan-based inference engine (such as llama.cpp) runs, the Vulkan interception library (vVulkan library) within the Windows virtual machine captures all API calls and forwards them to the host operating system (Host OS) for execution. Meanwhile, GPU sharding is used for graphics rendering tasks within virtual machines. The host OS partitions the physical GPU into multiple virtual GPUs and assigns them to different VMs to perform graphics rendering.

[0084] However, hybrid virtualization solutions that combine API forwarding and GPU sharding face a key issue: when all the video memory of the host operating system's physical GPUs is allocated to the virtual GPU, the Vulkan driver, acting as the API forwarding backend, loses access to video memory resources and must fall back to using system memory, creating a performance bottleneck. This situation is particularly pronounced when running high-performance computing tasks such as large model inference, as the lack of video memory significantly increases data transmission latency and reduces overall efficiency.

[0085] To address technical problems similar to those described above, embodiments of the present disclosure provide a method for managing video memory for GPU virtualization. A virtual machine transmits a page table root address corresponding to a target application running in the virtual machine to a host machine, wherein the virtual machine is configured with a virtual GPU, and the virtual GPU is allocated a dedicated video memory area. The video memory area corresponding to the target application belongs to the dedicated video memory area. In response to the target application calling a non-memory operation API, the virtual machine intercepts the call to the non-memory operation API and forwards the call to the host machine. In the host machine, a driver using the non-memory operation API accesses the video memory area corresponding to the target application based on the page table root address, executes the instructions of the non-memory operation API, and returns the processing results of the non-memory operation API instructions to the virtual machine. This enables a backend driver (e.g., a Vulkan driver) in the host machine to directly access the video memory resources allocated by the virtual machine. That is, the backend driver in the host machine can share and use the video memory resources allocated by the virtual machine, avoiding the performance bottleneck of being forced to use system memory due to insufficient video memory on the host machine. The disclosed embodiments not only improve the execution efficiency of API forwarding, especially reduce data transmission delay when processing compute-intensive applications (such as applications based on artificial intelligence models), but also enhance the isolation of video memory between different virtual machines, thereby optimizing GPU resource utilization in hybrid virtualization scenarios.

[0086] The following describes in detail the GPU virtualization memory management method provided by the embodiments of the present disclosure with reference to the accompanying drawings.

[0087] Figure 1 A flowchart of a method for managing video memory for GPU virtualization provided by an embodiment of the present disclosure is shown. In one possible implementation, the execution subject of the method for managing video memory for GPU virtualization may be a video memory management device for GPU virtualization. For example, the method for managing video memory for GPU virtualization may be executed by a terminal device or a server or other electronic device. The terminal device may be a user equipment (UE), a mobile device, a user terminal, a terminal, a handheld device, a computing device or an in-vehicle device, etc. In some possible implementations, the method for managing video memory for GPU virtualization may be implemented by a processor calling computer-readable instructions stored in a memory. For example, Figure 1 As shown, the GPU virtualization video memory management method includes steps S11 to S13.

[0088] In step S11, the virtual machine passes the page table root address corresponding to the target application running in the virtual machine to the host machine; wherein the virtual machine is configured with a virtual GPU, the virtual GPU is allocated a dedicated video memory area, and the video memory area corresponding to the target application belongs to the dedicated video memory area.

[0089] In step S12, in response to the target application calling a non-memory operation API, the virtual machine intercepts the call of the non-memory operation API and forwards the call to the host machine.

[0090] In step S13, in the host machine, the video memory area corresponding to the target application is accessed based on the page table root address through the driver of the non-memory operation class API, the instructions of the non-memory operation class API are executed, and the processing results of the instructions of the non-memory operation class API are returned to the virtual machine.

[0091] The disclosed embodiments can be applied in GPU virtualization scenarios, such as cloud computing, virtual desktop infrastructure (VDI), or artificial intelligence computing environments. By enabling the host machine to directly access the video memory resources allocated to the virtual machine, the disclosed embodiments can ensure efficient execution of computing tasks even when physical video memory resources are limited, avoiding performance degradation caused by insufficient video memory. Furthermore, the disclosed embodiments maintain video memory isolation between virtual machines, improving resource utilization while ensuring the security and stability of tasks across different virtual machines, providing a more flexible video memory management solution for GPU virtualization.

[0092] In one possible implementation, the method can be applied to a hybrid GPU virtualization scenario. A hybrid GPU virtualization scenario refers to a technical solution that simultaneously utilizes at least two GPU virtualization technologies (such as pass-through, slicing virtualization, and hardware virtualization) within a virtualized environment. This hybrid GPU virtualization scenario can be used in cloud computing or virtual desktop infrastructure, providing differentiated GPU resource allocation and access mechanisms for compute-intensive tasks (such as AI inference) and graphics rendering, thereby optimizing overall resource utilization and task execution efficiency.

[0093] In one possible implementation, the method is applied to a scenario where API forwarding and GPU sharding virtualization are mixed.

[0094] API forwarding forwards graphics / compute API calls (such as Vulkan and CUDA) from a guest VM to the host OS for execution. This allows multiple VMs to share a physical GPU, but performance depends on forwarding efficiency. GPU slicing virtualization divides a physical GPU into multiple virtual GPUs (vGPUs), with each VM exclusively using one vGPU. This is suitable for graphics rendering tasks, but video memory resources may be fully allocated to the vGPU, leaving the host OS without any available video memory.

[0095] In a hybrid scenario of API forwarding and GPU sharding virtualization, API forwarding can be used for compute-intensive tasks (such as AI inference), and GPU sharding virtualization can be used for graphics rendering tasks (such as 3D rendering).

[0096] In related technologies, in scenarios where API forwarding and GPU sharding virtualization are combined, after the host allocates all its physical video memory to the virtual GPU, the host itself has no remaining video memory. In this situation, the driver (such as the Vulkan driver), acting as the API forwarding backend, cannot allocate video memory and must fall back to using system memory, resulting in a significant performance degradation (because system memory bandwidth and latency are far inferior to video memory). This insufficient video memory can lead to performance bottlenecks for compute-intensive tasks (such as large model inference).

[0097] In this implementation, the virtual machine can pass the page table root address of the target application (such as an AI inference application) to the host machine. The host machine's backend driver (such as the Vulkan driver) does not need to allocate video memory itself, but can directly use the video memory allocated by the virtual machine, avoiding system memory bottlenecks.

[0098] This implementation allows the host machine to directly access the virtual machine's video memory, avoiding performance degradation caused by insufficient video memory. Furthermore, this implementation is compatible with mixed workloads involving compute-intensive workloads and graphics rendering, optimizing GPU utilization.

[0099] In one possible implementation, before transferring the page table root address corresponding to the target application running in the virtual machine to the host machine, the method further includes: loading a GPU driver in the host machine and creating multiple virtual GPUs, and allocating dedicated video memory areas to each of the multiple virtual GPUs; and the host machine transparently transferring the virtual GPUs to the virtual machine.

[0100] In this implementation, the host can load the physical GPU driver and an intermediate driver that supports virtualization (such as the mdev (Mediated Device) driver). This allows the host to identify and manage the physical GPU and provide support for the subsequent creation of a virtual GPU.

[0101] In this implementation, GPU slicing virtualization technology can be used to partition a physical GPU into multiple logical virtual GPUs. Each virtual GPU can be an independent logical device with its own device identifier (such as a PCIe virtual function (VF)) and resource quota. For example, a physical GPU (with 16GB of video memory) can be partitioned into four virtual GPUs, each with 4GB of video memory.

[0102] In this implementation, the host machine can allocate a fixed memory region to each virtual GPU (e.g., 0-4GB for virtual GPU 1, 4-8GB for virtual GPU 2, and so on). In this implementation, the memory regions can be exclusive, meaning that the memory of one virtual GPU cannot be occupied by other virtual GPUs. This ensures that memory usage between different virtual GPUs does not interfere with each other, preventing data leakage or contention.

[0103] In this implementation, the host machine can transparently transmit the virtual GPU to the virtual machine. For example, PCIe passthrough or intermediary passthrough can be used to directly assign the virtual GPU device to the virtual machine. The virtual machine can be a Windows virtual machine, for example, and is not limited here. After the virtual machine starts, it loads the corresponding virtual GPU driver and can directly use the virtual GPU's video memory and computing resources.

[0104] In a possible implementation, the sum of the capacities of the dedicated video memory areas of the multiple virtual GPUs is equal to the total video memory capacity of the physical GPU.

[0105] In this implementation, the entire physical GPU's video memory can be fully partitioned and allocated to each virtual GPU, leaving no unused space. This maximizes memory utilization and avoids resource waste caused by memory fragmentation or unallocated areas. For example, a physical GPU with 16GB of video memory can be partitioned into four virtual GPUs, each with 4GB of memory, ensuring that all video memory is effectively utilized.

[0106] In related technologies, this memory allocation scheme results in the host machine being unable to allocate memory for tasks like API forwarding. However, by adopting the GPU virtualization memory management method provided by the disclosed embodiments, the virtual machine's allocated memory is shared, allowing the host machine's backend driver to directly access this memory area, thus maintaining high performance even when the memory is fully allocated. This not only improves memory utilization but also resolves performance bottlenecks in hybrid virtualization scenarios.

[0107] In an embodiment of the present disclosure, a page table root address corresponding to a target application running in a virtual machine can be transferred to a host machine. The virtual machine is configured with a virtual GPU, and the virtual GPU is allocated a dedicated video memory area, and the video memory area corresponding to the target application belongs to the dedicated video memory area.

[0108] The page table is the core data structure used by the operating system to manage memory, recording the mapping between virtual addresses and physical addresses. The page table root (PC Root) is the starting address of the page table. This root is used to traverse the entire page table to find the physical address corresponding to the application's video memory. In GPU virtualization, the physical address of the video memory allocated within the virtual machine by the target application (such as an AI inference program) is managed by the virtual machine's page table.

[0109] After obtaining the page table root address corresponding to the target application, the host can directly resolve the physical location of the target application's video memory within the virtual machine, eliminating the need to reallocate video memory on the host. Therefore, when all host video memory has been allocated to the virtual GPU, the host backend driver (such as the Vulkan driver) can access the virtual machine's video memory through this address, avoiding inefficient use of system memory.

[0110] In one possible implementation, the target application includes a computationally intensive application.

[0111] Compute-intensive applications typically require continuous, extensive parallel computations, placing high demands on both processor and video memory performance. This implementation leverages a memory sharing mechanism to enable compute-intensive applications to efficiently access video memory resources within a virtualized environment, avoiding performance bottlenecks caused by insufficient host machine video memory.

[0112] In one possible implementation, the computationally intensive application includes an application based on an artificial intelligence model.

[0113] Applications based on artificial intelligence models (such as those based on large language models or computer vision models) have particularly prominent demands on video memory capacity and bandwidth, and require efficient management of large-scale parameters and input data. In this implementation, through the video memory sharing mechanism, the video memory allocated to AI workloads in the virtual machine can be directly used by the host machine, avoiding the performance degradation in traditional virtualization. Applications based on artificial intelligence models (such as inference engine applications using the Vulkan backend) usually need to frequently call computing APIs and rely on video memory to store intermediate results. This implementation ensures that AI tasks can still achieve near-native computing speeds in a virtualized environment by forwarding non-memory operation APIs to the host machine for execution while sharing video memory access rights.

[0114] In one example, a large model involved in an artificial intelligence model-based application may include a large model running on llama.cpp.

[0115] In one possible implementation, the method further includes: the virtual machine sends the identifier of the virtual machine and the process identifier of the target application to the host machine; in the host machine, the video memory area corresponding to the target application is accessed based on the page table root address through the driver of the non-memory operation class API, and the instructions of the non-memory operation class API are executed, including: in the host machine, the video memory area corresponding to the target application is accessed based on the identifier of the virtual machine, the process identifier of the target application and the page table root address through the driver of the non-memory operation class API, and the instructions of the non-memory operation class API are executed.

[0116] In this implementation, the virtual machine can also synchronize its unique identifier (VM ID) and the process identifier (PID) of the target application to the host machine. The VM ID can be used to determine the target VM instance, and the PID can be used to locate a specific application process within the VM. The page table root address can be used to resolve the physical mapping of the video memory for that process. This implementation can solve the video memory addressing problem in a multi-VM, multi-process environment. For example, in a cloud platform running multiple AI inference instances simultaneously, it enables the host machine to accurately associate API calls with corresponding video memory resources.

[0117] As an example of this implementation, the virtual machine can actively obtain the process identifier of the target application when the virtualization layer (such as the virtual Vulkan library) first establishes a connection with the target application, and forward it to the host machine together with the unique identifier of the virtual machine.

[0118] In one possible implementation, within a virtual machine, when the vVulkan library (virtual Vulkan library) first establishes a connection with a Vulkan application, it can proactively obtain the process identifier of that application and, combined with the unique identifier of the current virtual machine, forward it to the host. This provides the host with process-level location capabilities, enabling the host's Vulkan driver to correctly associate subsequently received API calls with the specific virtual machine process memory context.

[0119] In the disclosed embodiments, when a target application running in a virtual machine calls a non-memory operation API, the virtual machine can capture these API calls in real time, extract their execution context (e.g., parameters, queue information, and synchronization status), and forward them to the host machine. Upon receiving the API calls, the host machine can submit the API instructions to the physical GPU driver for execution based on the pre-obtained page table root address.

[0120] In the embodiments of the present disclosure, a memory operation API may refer to an API that directly manages underlying memory resources such as video memory allocation, release, or data migration; a non-memory operation API may refer to an API that performs functions such as computing tasks but does not involve video memory management.

[0121] In one possible implementation, the virtual machine intercepts the call of the non-memory operation class API and forwards it to the host machine in response to the target application calling the non-memory operation class API, including: the virtual machine intercepts the call of the non-memory operation class API through the virtualization layer and forwards it to the host machine in response to detecting that the target application calls the non-memory operation class API through the virtualization layer.

[0122] Among them, the virtualization layer (such as the vVulkan Library) is an intermediate interception layer running in the virtual machine, which can be used to monitor and manage the target application's calls to the GPU API. In some application scenarios, the virtualization layer can also be called the API virtualization layer, virtualization middleware, etc., which are not limited here. The virtualization layer can distinguish and process different types of API requests: for memory operation APIs (such as APIs for video memory allocation / release), they can be directly handed over to the virtual machine for local processing and record video memory mapping information; for non-memory operation APIs (such as APIs related to computing tasks), they can be intercepted and forwarded to the host machine for execution. Through this mechanism, the virtualization layer avoids the host machine being forced to use inefficient system memory due to insufficient video memory, thereby significantly improving GPU resource utilization and computing performance in hybrid virtualization scenarios.

[0123] In one possible implementation, the method further includes: the virtual machine responds to the target application calling a memory operation class API, processes the instructions of the memory operation class API in the virtual machine, and records the updated page table root address corresponding to the target application; the virtual machine passes the page table root address corresponding to the target application running in the virtual machine to the host machine, including: the virtual machine responds to the page table root address corresponding to the target application being updated, and passes the updated page table root address corresponding to the target application to the host machine.

[0124] In this implementation, when a target application calls a memory operation API, the instructions are processed directly by the virtual GPU driver within the virtual machine, without being forwarded to the host machine. Because the virtual machine doesn't intercept and forward these memory operation APIs, the host machine's GPU driver (such as the Vulkan driver) doesn't directly handle operations related to the target application's video memory management, thus avoiding the performance overhead and complexity incurred by host memory management intervention.

[0125] As an example of this implementation, a kernel mode driver (KMD) of the virtual GPU may be used to record a process identifier of the target application and a page table root address corresponding to the target application.

[0126] The page table root address corresponding to the target application can be updated after memory allocation or memory release. In this implementation, the virtual machine can respond to the update of the page table root address corresponding to the target application by transmitting the updated page table root address to the host machine so that the host machine can access the correct video memory area.

[0127] In a possible implementation, the memory operation API includes: an API for memory allocation; and / or an API for memory release.

[0128] In this implementation, memory operation APIs can include two types of APIs: APIs for video memory allocation (such as Vulkan's vkAllocateMemory), which are responsible for allocating dedicated video memory areas for applications within the virtual machine; and APIs for video memory release (such as Vulkan's vkFreeMemory), which are used to reclaim allocated video memory resources. The instructions of these memory operation APIs can be completely processed locally by the virtual GPU driver within the virtual machine, including updating the video memory page table and binding physical video memory, without requiring host intervention. This significantly reduces the memory management overhead of the virtualization layer. At the same time, a page table root address sharing mechanism ensures that the host correctly maps the virtual machine's allocated video memory physical addresses when executing computational instructions.

[0129] In a possible implementation, the method further includes: for the non-computationally intensive application running in the virtual machine, executing, through a virtual GPU driver in the virtual machine, instructions of a graphics API called by the non-computationally intensive application.

[0130] In this implementation, non-computationally intensive applications running in a virtual machine can be directly processed by the virtual GPU driver within the virtual machine. For example, when a non-computationally intensive application calls a graphics API like Direct3D, the virtual GPU driver within the virtual machine can directly take over the relevant instructions and submit the rendering commands to the virtual GPU assigned to the virtual machine for execution. This processing method maintains the native nature of the graphics rendering process and avoids the additional overhead incurred by API forwarding.

[0131] In this implementation, non-computationally intensive applications can directly access dedicated video memory areas through the virtual GPU driver, while computationally intensive applications can access video memory resources through a shared memory mechanism. This ensures both stable frame rates for graphics rendering and efficient completion of computational tasks, improving resource allocation and performance across different workload types in a virtualized environment.

[0132] In a possible implementation, the non-computation-intensive application includes a graphics rendering application.

[0133] In this implementation, non-computationally intensive applications may include graphics rendering applications, such as 3D games, CAD (Computer-Aided Design) tools, and video editing software. These applications primarily perform real-time image rendering and visual effects processing by calling graphics APIs such as Direct3D and OpenGL. Their performance characteristics are characterized by high-frequency calls to the GPU graphics pipeline (such as geometry processing, rasterization, and pixel shading) rather than large-scale parallel computing.

[0134] In this implementation, graphics rendering applications are treated differently from compute-intensive applications (such as AI inference applications). Graphics API calls are directly processed by the virtual GPU driver within the virtual machine, bypassing the host machine's API forwarding path. Rendering tasks are performed using dedicated GPU resources allocated by virtualization, thereby ensuring low latency and high frame rate for graphics processing.

[0135] The GPU virtualization video memory management method provided in the embodiments of the present disclosure can be applied to technical fields such as GPU virtualization and Virtual Desktop Infrastructure (VDI), and is not limited here.

[0136] The following describes the GPU virtualization memory management method provided by the embodiments of the present disclosure through a specific application scenario. Figure 2 A schematic diagram of a method for managing video memory for GPU virtualization provided by an embodiment of the present disclosure is shown. Figure 2 This article describes the interaction process of video memory sharing and API forwarding between the host OS and the guest OS in a GPU virtualization scenario that combines API forwarding and GPU sharding virtualization.

[0137] exist Figure 2In a virtual machine (VM), the host can run a physical GPU driver (such as the Vulkan driver), which handles non-memory APIs forwarded from the VM and directly accesses the VM's allocated video memory resources. This physical GPU driver can include the mdev driver. The VMM (Virtual Machine Monitor) coordinates communication between the host and VMs, including API forwarding and video memory sharing.

[0138] A virtual machine can configure a virtual GPU (vGPU) and allocate dedicated video memory areas. The virtual GPU driver (vGPUDriver) manages video memory allocation within the virtual machine (for example, Direct3D graphics rendering tasks). The virtual Vulkan library (vVulkan Library) monitors API calls from Vulkan applications within the virtual machine and distinguishes between memory-related and non-memory-related Vulkan APIs. For example, memory-related APIs (such as vkAllocateMemory) called by Vulkan applications within the virtual machine can be processed locally by the virtual GPU driver, which records the page table root address of the process corresponding to the Vulkan application. The virtual machine can pass the page table root address and process identifier of the Vulkan application's process to the host machine. Non-memory-related APIs can be intercepted and forwarded to the host machine for execution. The host machine can directly access the dedicated video memory area allocated to the virtual GPU in the virtual machine based on the page table root address of the Vulkan application's process and return the processing results of the non-memory-related APIs to the virtual machine. Furthermore, graphics APIs called by Direct3D applications and other applications can be directly processed by the virtual GPU driver within the virtual machine.

[0139] In this application scenario, the host machine can directly use the video memory allocated to the virtual machine, avoiding the performance bottleneck of using system memory due to insufficient physical video memory in the host machine. In addition, compute-intensive tasks (such as AI inference) are efficiently executed through API forwarding, and graphics rendering tasks are handled locally by the virtual GPU, which can improve overall resource utilization.

[0140] It is understood that the above-mentioned various method embodiments mentioned in this disclosure can be combined with each other to form combined embodiments without violating the principle logic. Due to space limitations, this disclosure will not go into details. It is understood by those skilled in the art that in the above-mentioned methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.

[0141] In addition, the present disclosure also provides a GPU virtualization memory management device, a non-volatile computer-readable storage medium, and a computer program product, all of which can be used to implement any GPU virtualization memory management method provided by the present disclosure. The corresponding technical solutions and technical effects can be found in the corresponding records in the method section and will not be repeated here.

[0142] Figure 3 FIG. 1 is a block diagram of a GPU virtualization memory management device provided by an embodiment of the present disclosure. Figure 3 As shown, the GPU virtualized video memory management device includes:

[0143] a transfer module 31 configured to transfer the page table root address corresponding to the target application running in the virtual machine to the host machine; wherein the virtual machine is configured with a virtual GPU, the virtual GPU is allocated a dedicated video memory area, and the video memory area corresponding to the target application belongs to the dedicated video memory area;

[0144] A forwarding module 32, configured for the virtual machine to intercept the call of the non-memory operation class API in response to the target application calling the non-memory operation class API and forward the call to the host machine;

[0145] The first processing module 33 is used to access the video memory area corresponding to the target application based on the page table root address through the driver of the non-memory operation class API in the host machine, execute the instructions of the non-memory operation class API, and return the processing results of the instructions of the non-memory operation class API to the virtual machine.

[0146] In one possible implementation,

[0147] The apparatus further includes: a sending module, configured for the virtual machine to send an identifier of the virtual machine and a process identifier of the target application to the host machine;

[0148] The first processing module 33 is used to: in the host machine, access the video memory area corresponding to the target application based on the identifier of the virtual machine, the process identifier of the target application and the page table root address through the driver of the non-memory operation class API, and execute the instructions of the non-memory operation class API.

[0149] In a possible implementation, the apparatus further includes:

[0150] A creation and allocation module, configured to load a GPU driver in the host machine and create multiple virtual GPUs, and allocate dedicated video memory areas to each of the multiple virtual GPUs;

[0151] The transparent transmission module is used for the host machine to transparently transmit the virtual GPU to the virtual machine.

[0152] In a possible implementation, the sum of the capacities of the dedicated video memory areas of the multiple virtual GPUs is equal to the total video memory capacity of the physical GPU.

[0153] In a possible implementation, the forwarding module 32 is configured to:

[0154] In response to detecting, through the virtualization layer, that the target application program calls a non-memory operation class API, the virtual machine intercepts the call of the non-memory operation class API through the virtualization layer and forwards the call to the host machine.

[0155] In one possible implementation,

[0156] The apparatus further includes: a second processing module configured to, in response to the target application calling a memory operation class API, process instructions of the memory operation class API in the virtual machine and record an updated page table root address corresponding to the target application;

[0157] The transfer module 31 is configured to: in response to an update of a page table root address corresponding to the target application, the virtual machine transfers the updated page table root address corresponding to the target application to the host machine.

[0158] In one possible implementation, the memory operation API includes:

[0159] API for memory allocation;

[0160] and / or,

[0161] API for memory release.

[0162] In one possible implementation, the target application includes a computationally intensive application.

[0163] In one possible implementation, the computationally intensive application includes an application based on an artificial intelligence model.

[0164] In a possible implementation, the apparatus further includes:

[0165] An execution module is configured to execute, for the non-computation-intensive application running in the virtual machine, instructions of a graphics API called by the non-computation-intensive application through a virtual GPU driver in the virtual machine.

[0166] In a possible implementation, the non-computation-intensive application includes a graphics rendering application.

[0167] In a possible implementation, the device is applied to a scenario where API forwarding and GPU sharding virtualization are mixed.

[0168] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. Its specific implementation and technical effects can refer to the description of the above method embodiments. For the sake of brevity, they will not be repeated here.

[0169] An embodiment of the present disclosure further provides a GPU virtualized video memory management device, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above method.

[0170] An embodiment of the present disclosure further provides a non-volatile computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0171] An embodiment of the present disclosure further provides a computer program product, including a computer program, or a non-volatile computer-readable storage medium carrying the computer program, wherein the computer program implements the steps of the above method when executed by a processor.

[0172] Figure 4 1 is a block diagram of a GPU virtualization memory management device according to an exemplary embodiment. For example, the device 1900 can be provided as a server or terminal device. Figure 4 The apparatus 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions, such as an application, that can be executed by the processing component 1922. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute the instructions to perform the above-described method.

[0173] The device 1900 may also include a power supply component 1926 configured to perform power management of the device 1900, a wired or wireless network interface 1950 configured to connect the device 1900 to a network, and an input / output interface 1958 (I / O interface). The device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server 2003. TM , MacOS X TM , Unix TM , Linux TM , FreeBSD TM or similar.

[0174] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by the processing component 1922 of the apparatus 1900 to perform the above-described method.

[0175] A computer-readable storage medium can be a tangible device that can hold and store programs / instructions used by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or raised structure within a groove on which instructions are stored, and any suitable combination thereof. As used herein, a computer-readable storage medium is not to be construed as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted via wires.

[0176] The computer programs (or computer-readable program instructions) described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage medium in the respective computing / processing device for storage.

[0177] The computer program (or computer program instructions) used to perform the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, the state information of the computer-readable program instructions is used to personalize an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), so that the electronic circuit can execute the computer-readable program instructions, thereby implementing various aspects of the present disclosure.

[0178] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0179] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0180] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0181] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and the part of the module, program segment or instruction contains one or more executable instructions for realizing the prescribed logical function. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the prescribed function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0182] The computer program product may be implemented in hardware, software, or a combination thereof. In one embodiment, the computer program product is implemented as a computer storage medium. In another embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0183] The above description of the various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced with each other and will not be repeated herein for the sake of brevity.

[0184] If the technical solutions of the embodiments of the present disclosure involve personal information, the products applying the technical solutions of the embodiments of the present disclosure have clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solutions of the embodiments of the present disclosure involve sensitive personal information, the products applying the technical solutions of the embodiments of the present disclosure have obtained the individual's separate consent before processing the sensitive personal information, and at the same time meet the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that they agree to the collection of their personal information; or on the personal information processing device, when the personal information processing rules are notified by obvious signs / information, the individual's authorization is obtained through pop-up information or by asking the individual to upload their personal information. The personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0185] While various embodiments of the present disclosure have been described above, the above descriptions are illustrative, non-exhaustive, and not intended to be limiting of the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or improvements to existing technologies, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for managing video memory for GPU virtualization, characterized in that: include: The virtual machine transmits a page table root address corresponding to a target application running in the virtual machine to a host machine; wherein the virtual machine is configured with a virtual GPU, the virtual GPU is allocated a dedicated video memory area, and the video memory area corresponding to the target application belongs to the dedicated video memory area; In response to the target application calling a non-memory operation class API, the virtual machine intercepts the call of the non-memory operation class API and forwards the call to the host machine, wherein the non-memory operation class API is an API that performs computing tasks but does not involve video memory management; In the host machine, the graphics memory area corresponding to the target application is accessed based on the page table root address through the driver of the non-memory operation class API, the instructions of the non-memory operation class API are executed, and the processing results of the instructions of the non-memory operation class API are returned to the virtual machine.

2. The method according to claim 1, characterized in that The method further includes: the virtual machine sending an identifier of the virtual machine and a process identifier of the target application to the host machine; The method of accessing the video memory area corresponding to the target application based on the page table root address through the driver of the non-memory operation class API in the host machine and executing the instructions of the non-memory operation class API includes: accessing the video memory area corresponding to the target application based on the identifier of the virtual machine, the process identifier of the target application and the page table root address through the driver of the non-memory operation class API in the host machine and executing the instructions of the non-memory operation class API.

3. The method according to claim 1, characterized in that Before transferring the page table root address corresponding to the target application running in the virtual machine to the host machine, the method further includes: Loading a GPU driver in the host machine and creating multiple virtual GPUs, and allocating dedicated video memory areas to each of the multiple virtual GPUs; The host machine transparently transmits the virtual GPU to the virtual machine.

4. The method according to claim 3, characterized in that The sum of the capacities of the dedicated video memory areas of the multiple virtual GPUs is equal to the total video memory capacity of the physical GPU.

5. The method according to any one of claims 1 to 4, characterized in that The virtual machine intercepts the call of the non-memory operation class API in response to the target application calling the non-memory operation class API and forwards the call to the host machine, including: In response to detecting, through the virtualization layer, that the target application program calls a non-memory operation class API, the virtual machine intercepts the call of the non-memory operation class API through the virtualization layer and forwards the call to the host machine.

6. The method according to any one of claims 1 to 4, characterized in that The method further includes: the virtual machine, in response to the target application calling a memory operation class API, processing instructions of the memory operation class API in the virtual machine, and recording an updated page table root address corresponding to the target application, wherein the memory operation class API refers to an API that directly manages underlying memory resources such as video memory allocation, release, or data migration; The virtual machine transmits the page table root address corresponding to the target application running in the virtual machine to the host machine, including: the virtual machine transmits the updated page table root address corresponding to the target application to the host machine in response to the page table root address corresponding to the target application being updated.

7. The method according to claim 6, characterized in that The memory operation API includes: API for memory allocation; and / or, API for memory release.

8. The method according to claim 1, characterized in that The target applications include computationally intensive applications.

9. The method according to claim 8, characterized in that The computationally intensive applications include applications based on artificial intelligence models.

10. The method according to claim 1, characterized in that The method further comprises: For the non-computation-intensive application program running in the virtual machine, instructions of the graphics API called by the non-computation-intensive application program are executed through the virtual GPU driver in the virtual machine.

11. The method according to claim 10, characterized in that The non-computation-intensive application programs include graphics rendering applications.

12. The method according to claim 1, characterized in that The method is applied to a scenario where API forwarding and GPU sharding virtualization are mixed.

13. A GPU virtualization memory management device, characterized in that: include: a transfer module, configured for the virtual machine to transfer a page table root address corresponding to a target application running in the virtual machine to a host machine; wherein the virtual machine is configured with a virtual GPU, the virtual GPU is allocated a dedicated video memory area, and the video memory area corresponding to the target application belongs to the dedicated video memory area; a forwarding module, configured for the virtual machine to intercept the call of the non-memory operation class API in response to the target application calling the non-memory operation class API and forward the call to the host machine, wherein the non-memory operation class API is an API that performs computing tasks but does not involve video memory management; The first processing module is used to access the video memory area corresponding to the target application based on the page table root address through the driver of the non-memory operation class API in the host machine, execute the instructions of the non-memory operation class API, and return the processing results of the instructions of the non-memory operation class API to the virtual machine.

14. A GPU virtualized video memory management device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 12.

15. A non-volatile computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

16. A computer program product comprising a computer program, or a non-volatile computer-readable storage medium carrying a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

Images