GPU (Graphics Processing Unit) security sharing method for multiple virtual machines

Through the combination of the T-Hyper virtual machine management system and the virtio-GPU front-end, the problem of improper allocation of GPU resources between virtual machines is solved, efficient utilization and secure sharing of GPU resources is realized, and system performance and security are improved.

CN119065842AActive Publication Date: 2024-12-03NINGBO QIANCHUAN TECH CO LTD

Patent Information

Application Number
CN202411129877.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-16
Publication Date
2024-12-03
Estimated Expiration
2044-08-16

AI Technical Summary

Technical Problem

In the prior art, virtual machines or physical machines are usually allocated independent GPU resources, which makes it difficult to efficiently utilize GPU resources and poses security risks, such as malicious virtual machines stealing data or occupying too much resources, resulting in performance degradation or crashes of other virtual machines. .

Method used

It adopts components such as T-Hyper trusted virtual machine management system, management server virtual machine upper GPU backend, virtual image engine interface, upper GPU encoder and virtio-GPU frontend, and uses the intermediate layer image engine library and DMA direct memory access technology. Realizes virtualization and secure sharing of GPU resources, provides virtual GPU interface, and supports flexible allocation and efficient communication.

Benefits of technology

It improves the utilization rate and security of GPU resources, reduces management costs, ensures secure isolation and reasonable allocation of resources between virtual machines, and avoids resource waste and potential security threats.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119065842A_ABST
    Figure CN119065842A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of interface management, discloses a GPU (Graphics Processing Unit) security sharing method for multiple virtual machines, and solves the problem that the GPU resources are difficult to efficiently utilize due to the fact that the existing virtual machines or physical machines are usually allocated with independent GPU resources. According to the method, indirect access of the UOS application program to the GPU resources is realized through the middle-layer image engine library, the virtual image engine interface, the upper-layer GPU encoder and the virtio-GPU front-end and rear-end components, the access efficiency and performance are improved, the security and the flexibility of the GPU resources are ensured, a plurality of virtual GPU instances are created on the physical GPU through GPU virtualization, and the security and the flexibility of the GPU resources are improved. According to the method, the physical GPU resources are divided into the plurality of virtual slices, and each virtual slice is allocated to one virtual machine, so that the utilization rate of the GPU resources is improved, and the management cost is reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of interface management, and in particular to a GPU security sharing method for multiple virtual machines. Background Art

[0002] With the rapid development of science and technology, GPU, as a powerful accelerator chip, has been widely used in high-performance computing, artificial intelligence, machine learning and big data processing. The parallel computing capability of GPU enables it to show great advantages in complex data processing and algorithm acceleration. However, in practical applications, the effective management and secure sharing of GPU resources has become a problem that needs to be solved urgently.

[0003] In the traditional way, each virtual machine or physical machine is usually allocated independent GPU resources, which makes it difficult to efficiently use GPU resources. Especially during non-peak hours, GPU resources are often idle, resulting in resource waste. GPU devices themselves are expensive. For scenarios that require a large amount of computing resources, the purchase and maintenance costs are very high. In an environment where multiple virtual machines share GPU resources, there are serious security risks. Malicious virtual machines steal data from other virtual machines, or occupy too many GPU resources through illegal means, causing performance degradation or even crash of other virtual machines.

[0004] To solve the above problems, an innovative design is carried out based on the original GPU security sharing method for multiple virtual machines. Summary of the invention

[0005] The purpose of the present invention is to provide a GPU security sharing method for multiple virtual machines. By adopting the device to work, the problem that existing virtual machines or physical machines are usually allocated independent GPU resources, resulting in difficulty in efficient use of GPU resources is solved.

[0006] To achieve the above object, the present invention provides the following technical solution: a GPU security sharing method for multiple virtual machines, comprising:

[0007] T-Hyper: T-Hyper is a trusted virtual machine management system that is responsible for creating and managing server virtual machines and user virtual machines, ensuring secure isolation of virtual machines and reasonable allocation of resources.

[0008] Manage the upper GPU backend of the server virtual machine: responsible for managing and calling GPU resources, optimizing the execution of GPU tasks through the middle-layer image engine library, and using DMA direct memory access technology to efficiently access shared memory;

[0009] Virtual Image Engine Interface: Provides a set of upper-level library modules as a bridge for UOS user operating system applications to access GPU graphics processing functions. Through standard interface encapsulation, it reduces the application's dependence on the underlying GPU hardware and provides UOS applications with an access interface to upper-level graphics processing functions.

[0010] Upper GPU encoder: used to encode the data generated by the virtual image engine interface, convert it into a format suitable for network transmission, and send it to the upper GPU backend of the management server virtual machine for processing;

[0011] Virtual GPU upper layer interface: responsible for directly providing UOS applications with an interface to access underlying graphics processing functions, bypassing the traditional graphics stack, improving artwork access efficiency and performance;

[0012] virtio-GPU front-end: responsible for calling the middle-layer image engine library and accessing shared memory through DMA technology to cooperate with the virtio-GPU back-end to achieve virtualization of GPU resources and efficient communication and data transmission between SOS and GPU resources, supporting flexible allocation and sharing of GPU resources.

[0013] Further, the method includes the following three methods:

[0014] C1: Access GPU hardware through the basic graphics access interface;

[0015] C2: communicates with the GPU backend in SOS through the virtio-GPU frontend;

[0016] C3: Directly use GPU resources by calling the underlying GPU interface through the virtual GPU upper interface.

[0017] Furthermore, the specific steps of C1 are as follows:

[0018] C1-1: The UOS application calls the virtual image engine interface, which is responsible for calling the upper GPU encoder to encode the interface according to the specified rules and pass it to the upper GPU backend through the fast channel;

[0019] C1-2: The upper GPU backend is responsible for receiving the encoded request and decoding it, identifying the real image engine library and interface name, calling the corresponding image engine library according to the request, and performing actual GPU operations. It can support multiple image engine libraries and realize flexible scheduling and sharing of GPU resources.

[0020] Furthermore, the specific steps of C2 are as follows:

[0021] C2-1: Direct communication between virtio-GPU front-end and back-end: Used for direct communication between virtio-GPU front-end and GPU back-end in SOS, eliminating encoder conversion and interacting efficiently through standard kernel interfaces;

[0022] C2-2: Sending of data stream addresses and processing instructions: used by the virtio-GPU front end to send data stream addresses and processing instructions to the GPU back end in SOS, directly locate data and specify operations, avoid data copying, and improve the performance bottleneck when processing large data images simultaneously using the standard graphics library interface.

[0023] Furthermore, the specific steps of C3 are as follows:

[0024] C3-1:Virtio-GPU access mechanism, including: control flow: used to process the exchange of commands and status information; data flow: the virtual machine directly accesses the GPU data on the host machine without copying the data.

[0025] C3-2: 3D engine library and GPU underlying interface: used to directly access the GPU underlying interface and control the rendering and computing capabilities of the GPU.

[0026] Furthermore, the specific steps of C3 are as follows:

[0027] C3-3:Vi rtua l-OpenGL and GPU encoder: The GPU encoder is used to efficiently encode and serialize the API calls of the 3D engine to reduce the memory usage and complexity in the user operating system UOS. These serialized data are transmitted to the GPU backend in the service operating system SOS through vhost-dmabuf or network mechanism for decoding and execution.

[0028] Furthermore, the specific steps of C3 are as follows:

[0029] C3-4: ASG and fast IPC: ASG is a communication method that implements fast inter-process communication IPC through an acceleration channel and is used in scenarios that require low latency and high bandwidth.

[0030] Further, the method comprises the following first step:

[0031] D1: Separation of server virtual machines and user virtual machines: In the T-Hyper virtualization architecture, the server virtual machine SOS and the user virtual machine UOS are clearly separated in terms of GPU resource access, achieving efficient resource sharing and secure isolation.

[0032] Further, the method comprises the following second step:

[0033] D2: Fine-grained access control: The microkernel-based T-Hyper divides system resources into fine-grained categories using a capability-based approach.

[0034] Further, the method comprises the following third step:

[0035] D3: GPU address protection: To prevent potential DMA attacks, GPU addresses are protected by the SMMU system memory management unit or MPU memory protection unit hardware isolation mechanism.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] The present invention proposes a GPU security sharing method for multiple virtual machines. Existing virtual machines or physical machines are usually allocated independent GPU resources, which makes it difficult to efficiently utilize GPU resources. The present invention implements indirect access of UOS applications to GPU resources through components such as an intermediate image engine library, a virtual image engine interface, an upper GPU encoder, and a virtual-GPU front-end and back-end, thereby improving access efficiency and performance, and ensuring the security and flexibility of GPU resources. Multiple virtual GPU instances are created on a physical GPU through GPU virtualization, and physical GPU resources are divided into multiple virtual slices, each of which is allocated to a virtual machine. This method not only improves the utilization rate of GPU resources, but also reduces management costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is an overall diagram of the multi-virtual machine GPU sharing system method of the present invention;

[0039] Figure 2 It is a flowchart of the present invention for accessing GPU hardware using a basic graphics access interface;

[0040] Figure 3 It is a flowchart of accessing GPU resources through the virtio-GPU front end using the basic image access interface of the present invention;

[0041] Figure 4 This is a flowchart of directly calling GPU resources using a virtual GPU upper layer interface of the present invention. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] In order to further understand the content of the present invention, the present invention is described in detail in conjunction with the accompanying drawings.

[0044] See also Figure 1 , a GPU secure sharing method for multiple virtual machines, comprising:

[0045] T-Hyper: T-Hyper is a trusted virtual machine management system that is responsible for creating and managing server virtual machines and user virtual machines, ensuring secure isolation of virtual machines and reasonable allocation of resources.

[0046] Manage the upper GPU backend of the server virtual machine: responsible for managing and calling GPU resources, optimizing the execution of GPU tasks through the middle-layer image engine library, and using DMA direct memory access technology to efficiently access shared memory;

[0047] Virtual Image Engine Interface: Provides a set of upper-level library modules as a bridge for UOS user operating system applications to access GPU graphics processing functions. Through standard interface encapsulation, it reduces the application's dependence on the underlying GPU hardware and provides UOS applications with an access interface to upper-level graphics processing functions.

[0048] Upper GPU encoder: used to encode the data generated by the virtual image engine interface, convert it into a format suitable for network transmission, and send it to the upper GPU backend of the management server virtual machine for processing;

[0049] Virtual GPU upper layer interface: responsible for directly providing UOS applications with an interface to access underlying graphics processing functions, bypassing the traditional graphics stack, improving artwork access efficiency and performance;

[0050] virtio-GPU front-end: responsible for calling the middle-layer image engine library and accessing shared memory through DMA technology to cooperate with the virtio-GPU back-end to achieve virtualization of GPU resources and efficient communication and data transmission between SOS and GPU resources, supporting flexible allocation and sharing of GPU resources.

[0051] The present invention will be further described below in conjunction with the embodiments.

[0052] See also Figure 1, as mentioned above, a shared use method of a multi-virtual machine GPU sharing system includes the following steps: T-Hyper is a trusted virtual machine management system, responsible for creating and managing server virtual machines and user virtual machines, ensuring the security isolation of virtual machines and the reasonable allocation of resources; the upper GPU backend of the management server virtual machine: responsible for managing and calling GPU resources, optimizing the execution of GPU tasks through the middle-layer image engine library, and using DMA direct memory access technology to efficiently access shared memory; virtual image engine interface: providing a set of upper-layer library modules as a bridge for UOS user operating system applications to access GPU graphics processing functions, reducing the application's dependence on the underlying GPU hardware through standard interface encapsulation, and providing UOS applications with upper-layer graphics processing functions Access interface; Upper GPU encoder: used to encode the data generated by the virtual image engine interface, convert it into a format suitable for network transmission, and send it to the upper GPU backend of the management server virtual machine for processing; Virtual GPU upper interface: responsible for directly providing UOS applications with an interface to access the underlying graphics processing functions, bypassing the traditional graphics stack, improving artwork access efficiency and performance; Virtio-GPU front end: responsible for calling the middle-layer image engine library, and accessing shared memory through DMA technology to cooperate with the Virtio-GPU back end to achieve virtualization of GPU resources and efficient communication and data transmission between SOS and GPU resources, support flexible allocation and sharing of GPU resources, and improve graphics processing efficiency and data encoding efficiency.

[0053] Specifically, firstly, the T-Hyper virtual machine management system is installed and configured on the physical hardware. As the core of the entire virtualization environment, T-Hyper is responsible for creating and managing a server virtual machine (SOS) and multiple user virtual machines (UOS), ensuring that each virtual machine has independent resource allocation and security isolation. Then, the GPU driver and the intermediate image engine library are deployed in the SOS. The intermediate image engine library is responsible for optimizing the execution of GPU tasks and improving the efficiency of graphics processing. The DMA direct memory access technology is used to efficiently access and manage the shared memory area to achieve fast data transmission between the SOS and the UOS. Then, a virtual image engine interface component is deployed in the UOS as a bridge for the UOS application to access the GPU graphics processing function. The virtual image engine interface is encapsulated through a standard interface to reduce the application's dependence on the underlying GPU hardware, so that the upper-level GPU encoder encodes the data generated by the virtual image engine interface, converts it into a format suitable for network transmission, and sends it to the upper-level GPU backend in the SOS for processing. Therefore, the virtual GPU upper-level interface directly provides the UOS application with an interface to access the underlying graphics processing function, bypassing the traditional graphics stack, improving access efficiency and performance, and deploying the virtual image engine interface in the SOS. The virtual GPU front end serves as an efficient communication and data transmission interface between SOS and GPU resources. The virtual GPU front end calls the middle-layer image engine library and accesses shared memory through DMA technology to cooperate with the virtual GPU back end to realize the virtualization of GPU resources and the flexible allocation and sharing between SOS and GPU resources. Through the fine-grained management function of T-Hyper, the direct access rights of SOS to GPU hardware are ensured, and the direct access of UOS to GPU hardware is restricted. The virtual image engine interface and the upper-layer GPU encoder are used to realize indirect access of UOS applications to GPU resources to ensure security. Through the cooperation of the virtual GPU front and back ends, the flexible allocation and efficient sharing of GPU resources between SOS and multiple UOS are realized. Finally, the algorithms of the middle-layer image engine library and the upper-layer GPU encoder are continuously optimized to improve the graphics processing efficiency and data encoding efficiency, monitor and manage the usage of GPU resources, dynamically adjust resource allocation according to the actual needs of the virtual machine, and ensure the stability and optimization of the overall system performance.

[0054] See also Figure 2, access the GPU hardware through the basic graphics access interface; the UOS application calls the virtual image engine interface, which is responsible for calling the upper-level GPU encoder to encode the interface according to the specified rules and pass it to the upper-level GPU backend through a fast channel; the upper-level GPU backend is responsible for receiving the encoded request and decoding it, identifying the real image engine library and interface name, calling the corresponding image engine library according to the request, and performing actual GPU operations. It can support multiple image engine libraries and realize flexible scheduling and sharing of GPU resources.

[0055] Specifically, first, a request is initiated through the UOS application. When the user or the application in the UOS system needs to perform graphics processing or computing tasks, the virtual image engine interface will be called. These interfaces are a set of standardized APIs provided by the system for upper-level applications to encapsulate access to GPU hardware. Secondly, the virtual image engine interface is processed. After the virtual image engine interface receives the request from the UOS application, it will forward the request to the upper-level GPU encoder according to the type of request (such as OpenGL rendering, CUDA calculation, etc.) and parameters, and then encode the request through the upper-level GPU encoder. The upper-level GPU encoder is responsible for encoding the received request according to the predetermined encoding rules. The encoding process includes converting the interface name, parameters and other information in the request into a universal and easy-to-transmit format. The encoded data will contain enough information so that the upper-level GPU backend can decode and identify the specific operation requirements, and then pass the request through the fast IPC channel. The encoded request data is passed to the upper-level GPU backend through a fast IPC (inter-process communication) channel based on shared memory. This channel design ensures The high efficiency and low latency of data transmission are the key to achieving high-performance GPU access. The upper GPU backend is further decoded and executed. After receiving the encoded request data, the upper GPU backend performs a decoding operation to identify the specific image engine library (such as OpenGL library, CUDA library, etc.) and interface name, and then calls the corresponding image engine library according to the request content to perform actual GPU operations. This process supports multiple image engine libraries and realizes flexible scheduling and sharing of GPU resources. Finally, the processing result is returned. After the GPU operation is completed, the processing result will be returned to the UOS application in the opposite path. If the operation is successful, the application can continue to perform subsequent tasks. If it fails, the corresponding processing is performed according to the error information. Through the above process, applications in the UOS system can easily use basic graphics access interfaces (such as CUDA, OpenGL, etc.) to access GPU hardware without directly paying attention to the driver and underlying details of the physical GPU. This mechanism not only improves the compatibility and portability of the system, but also improves system performance by optimizing the call path and reducing the intermediate layer overhead.

[0056] See also Figure 3 , communicate and access with the GPU backend in SOS through the virtio-GPU front end, direct communication between the virtio-GPU front end and the back end: used for direct communication between the virtio-GPU front end and the GPU back end in SOS, abandoning the encoder conversion, and interacting efficiently through the standard kernel interface; sending data stream addresses and processing instructions: used for the virtio-GPU front end to send data stream addresses and processing instructions to the GPU back end in SOS, directly locate data and specify operations, avoid data copying, and improve the performance bottleneck when using the standard graphics library interface to simultaneously process large data images. Improve the performance bottleneck when using the standard graphics library interface to simultaneously process large data images.

[0057] Specifically, first, the UOS application initiates a request. When the application in the UOS user virtual machine needs to perform graphics processing tasks, it will send requests to the vi rt io-GPU front end through the basic image access interface (such as OpenGL, CUDA, etc.). These requests contain specific instructions for graphics processing and the required data stream addresses. Secondly, after the virtual image engine interface receives the request from the UOS application, it will forward the request to the vi rt io-GPU front end according to the type of request (such as OpenGL rendering, CUDA calculation, etc.) and parameters. After receiving the request, the vi rt io-GPU front end will not perform complex encoding and decoding operations, but directly package the data stream address and processing instructions in the request into the data packet format specified by the vi rt io protocol. These data packets contain all the information required for GPU operation, and then the vi rt io-GPU front end uses the standard vi The rtio interface sends the data packet to the GPU backend in the SOS on the host machine, so that after receiving the data packet, it parses the data stream address and processing instructions. It uses DMA direct memory access technology to directly read the corresponding image data from the physical memory of the host machine and pass this data to the physical GPU hardware for processing. DMA technology reduces CPU intervention and improves the efficiency of data transmission. Through the above process, the UOS user virtual machine can efficiently use the vi rtio-GPU front end to interact with the GPU resources on the host machine, thereby achieving fast access and efficient use of GPU resources, which not only improves the compatibility and portability of the system, but also improves system performance by reducing the overhead in data transmission and processing.

[0058] See also Figure 4, directly use GPU resources by calling the underlying GPU interface through the virtual GPU upper layer interface, Virt io-GPU access mechanism, including control flow for processing command and status information exchange and data flow for virtual machines to directly access GPU data on the host machine without copying data, 3D engine library and GPU underlying interface: used to directly access the GPU underlying interface and control the rendering and computing capabilities of the GPU, Virtua l-OpenGL and GPU encoder: efficiently encode and serialize the API calls of the 3D engine through the GPU encoder to reduce the memory usage and complexity in the user operating system UOS, and these serialized data are transmitted to the GPU backend in the service operating system SOS through vhost-dmabuf or network mechanism for decoding and execution, ASG and fast IPC: ASG is a communication method that realizes fast inter-process communication IPC through acceleration channels, which is used in scenarios requiring low latency and high bandwidth to improve the efficiency of data transmission and processing.

[0059] Specifically, first, the UOS application initiates a request. When the application in the UOS user virtual machine needs to perform graphics processing tasks, it sends requests to the virt io-GPU front end through the upper interface of the virtua l GPU. These requests contain specific instructions for graphics processing and the required data stream addresses. The user operating system UOS can directly send GPU instructions to the GPU hardware in the host operating system SOS without going through the mediation of the graphics processing library. It UOS communicates with the GPU back end in SOS through the standard virt io interface, and then virt The io-GPU protocol divides communication into control flow and data flow. The control flow is used to transmit GPU commands and status information, while the data flow is used to efficiently transmit the large amount of data required for GPU processing. In scenarios requiring low latency and high bandwidth, ASG (Accelerated Service Gateway) technology is used to establish a fast IPC (inter-process communication) channel. GPU commands and status information are quickly transmitted between UOS and SOS through the ASG channel, reducing transmission latency and bandwidth occupancy. In data stream transmission, ION technology is used to create a shared memory area, allowing UOS and SOS to directly access the same physical memory, thereby avoiding data duplication and additional memory overhead, reducing middle-layer overhead, and improving data transmission and processing efficiency, thereby meeting the needs of high-performance computing and graphics processing applications.

[0060] As mentioned above, a GPU security sharing method for multiple virtual machines includes the following steps: separation of server virtual machines and user virtual machines: in the T-Hyper virtualization architecture, a clear separation of the server virtual machine SOS and the user virtual machine UOS in GPU resource access is implemented, and efficient resource sharing and secure isolation are achieved. Fine-grained access control: microkernel-based T-Hyper divides system resources in a fine-grained manner through capability-based methods. GPU address protection: in order to prevent potential DMA attacks, the GPU address is protected by the SMMU system memory management unit or the MPU memory protection unit hardware isolation mechanism.

[0061] Specifically, the implementation of the GPU security sharing method is to build a virtualization architecture and use the microkernel-based Hypervisor as a virtualization platform. The platform supports the creation of multiple virtual machines, including a unique server virtual machine (SOS) and multiple user virtual machines (UOS). The GPU driver is installed and configured in the SOS to ensure that the SOS can directly access the GPU hardware address. Secondly, the GPU access permission control is carried out through the fine-grained management capability of Hypervisor to strictly limit the GPU access permission in the SOS. The UOS cannot directly access the GPU hardware address and can only access it through the virtualization interface Virt i o-GPU communicates with the GPU backend in SOS, and sets the GPU access security policy in SOS, including limiting the access capabilities of different applications to the GPU interface. Through capability level division, the application's access rights to different GPU access channels can be restricted, and the application can be restricted from directly accessing and operating the image engine library through the underlying interface. The GPU address is protected by the SMMU or MPU chip isolation mechanism. These hardware mechanisms can prevent DMA attacks and ensure the security and integrity of GPU memory. The SMMU / MPU is configured to limit access rights to the GPU address space to ensure that only authorized processes or devices can access the GPU memory. Then, through the virtualization interface, the Vir io-GPU backend is implemented in SOS, which is responsible for processing GPU access requests from UOS. The Vir io-GPU backend communicates with the Vir io-GPU front end in UOS through control flow and data flow. The control flow is used to process the exchange of commands and status information. The data flow allows the virtual machine to directly access the GPU data on the host machine. This is achieved through the shared memory mechanism without the need to copy data. Vir io The o-GPU front-end runs in UOS and provides a virtualized GPU access interface for applications in UOS. Applications send GPU commands and data through this interface to the Vir rt i o-GPU back-end in SOS for processing, thereby optimizing performance and scheduling resources. The GPU scheduler is implemented in SOS, which is responsible for managing and scheduling the use of GPU resources. The scheduler dynamically allocates GPU resources to each virtual machine based on the needs of the virtual machine and the available resources of the GPU, and then improves the utilization and performance of GPU resources by optimizing the virtualization interface and GPU scheduling algorithm. The GPU encoder can be used to efficiently encode and serialize the API calls of the 3D engine to reduce memory usage and complexity;At the same time, technologies such as ASG (Accelerated Service Gateway) are used to achieve fast inter-process communication (IPC), reduce transmission delay and bandwidth usage, and realize secure sharing of GPUs based on the microkernel Hypervisor, ensuring that the stability and security of SOS are not affected by potential risks in UOS. At the same time, fine-grained management and optimization technologies are used to improve the utilization and performance of GPU resources. ;

[0062] In traditional GPU virtualization, GPU is generally sliced ​​to allocate graphics card time slices to virtual machines. Some GPU scheduling algorithms adopt priority-based methods to set the priority of each task and schedule high-priority tasks first. However, these settings are generally static and cannot dynamically adjust the use of GPU resources by tasks according to the load and operation of the GPU. Therefore, a secure sharing method is combined to ensure that each virtual machine has clear permission control and isolation mechanisms when accessing GPU resources. GPU resources are dynamically allocated according to the real-time needs of the virtual machine and the load of the system. Dynamic scheduling of GPU resources is deployed in a virtualized environment, including a GPU resource manager, a virtual machine monitor (VMM) and a credit point management module. The GPU resource manager is responsible for receiving GPU resource requests from virtual machines and allocating resources according to weights, capabilities and credit point mechanisms. The virtual machine monitor (VMM) is responsible for monitoring the operating status and GPU resource usage of each virtual machine and providing necessary information to the GPU resource manager. The credit point management module is responsible for dynamically adjusting the credit points of each task according to the task execution status and preset rules, and setting the priority of the task accordingly. When a virtual machine is created or configured, a weight and capability value are assigned to each virtual machine.The weight value determines the relative priority and resource allocation ratio of virtual machines when sharing GPU resources. For virtual machine A with a weight of 3 and virtual machine B with a weight of 1, when they share a physical GPU, the GPU resource manager will allocate time slices to the two virtual machines at a ratio of 3:1. The capacity value limits the maximum frequency of GPU usage by the virtual machine per unit time. Even if there is no competition from other virtual machines, virtual machines with lower capacity values ​​will not obtain GPU resources beyond their capacity limits. Based on the priority mechanism of credit points, initial credit points are allocated to each task, and different credit points per unit time are set according to the priority and execution requirements of the task. When the task is executed, its credit points will be dynamically consumed at the rate of credit point consumption per unit time. According to the remaining credit points of the task, the credit point management module divides the priority of the task into four levels: high, medium, low, and idle. The higher the priority of the task, the less likely its GPU resources are to be occupied by other tasks. When a high-priority task requires GPU resources, the GPU resource manager will give priority to filling it. Meet its needs, even if this means interrupting the currently executing low-priority task, dynamic scheduling and resource optimization, real-time monitoring of GPU resource usage and task execution status in the system, for tasks with a higher response level in the vehicle system, such as the collection and processing of images for automatic parking, dynamically increase the weight and capability value of the relevant virtual machine to ensure that it can obtain sufficient GPU resources first, dynamically adjust credit point allocation and priority setting according to task execution and system load, so as to optimize the utilization efficiency of GPU resources and system performance, ensure the safe isolation between virtual machines and the reasonable allocation of GPU resources through the collaborative work of the virtual machine monitor and the GPU resource manager, monitor and prevent illegal resource occupation and malicious attacks between virtual machines, and ensure the stability and security of the system. Through the above specific implementation methods, the virtualization-based GPU resource dynamic scheduling system proposed in the present invention can effectively realize efficient, flexible and secure dynamic scheduling of GPU resources, and meet the needs of different virtual machines and application scenarios for GPU resources.

[0063] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0064] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A multi-virtual machine GPU sharing system, characterized by: include: T-Hyper: T-Hyper is a trusted virtual machine management system that is responsible for creating and managing server virtual machines and user virtual machines, ensuring secure isolation of virtual machines and reasonable allocation of resources. Manage the upper GPU backend of the server virtual machine: responsible for managing and calling GPU resources, optimizing the execution of GPU tasks through the middle-layer image engine library, and using DMA direct memory access technology to efficiently access shared memory; Virtual Image Engine Interface: Provides a set of upper-level library modules as a bridge for UOS user operating system applications to access GPU graphics processing functions. Through standard interface encapsulation, it reduces the application's dependence on the underlying GPU hardware and provides UOS applications with an access interface to upper-level graphics processing functions. Upper GPU encoder: used to encode the data generated by the virtual image engine interface, convert it into a format suitable for network transmission, and send it to the upper GPU backend of the management server virtual machine for processing; Virtual GPU upper layer interface: responsible for directly providing UOS applications with an interface to access underlying graphics processing functions, bypassing the traditional graphics stack, improving artwork access efficiency and performance; virtio-GPU front-end: responsible for calling the middle-layer image engine library and accessing shared memory through DMA technology to cooperate with the virtio-GPU back-end to achieve virtualization of GPU resources and efficient communication and data transmission between SOS and GPU resources, supporting flexible allocation and sharing of GPU resources.

2. The method for sharing and using a multi-virtual machine GPU sharing system according to claim 1, characterized in that: There are three methods: C1: Access GPU hardware through the basic graphics access interface; C2: communicates with the GPU backend in SOS through the virtio-GPU frontend; C3: Directly use GPU resources by calling the underlying GPU interface through the virtual GPU upper interface.

3. The method for sharing and using a multi-virtual machine GPU sharing system according to claim 2, characterized in that: The specific steps of C1 are as follows: C1-1: The UOS application calls the virtual image engine interface, which is responsible for calling the upper GPU encoder to encode the interface according to the specified rules and pass it to the upper GPU backend through the fast channel; C1-2: The upper GPU backend is responsible for receiving the encoded request and decoding it, identifying the real image engine library and interface name, calling the corresponding image engine library according to the request, and performing actual GPU operations. It can support multiple image engine libraries and realize flexible scheduling and sharing of GPU resources.

4. The method for sharing and using a multi-virtual machine GPU sharing system according to claim 2, characterized in that: The specific steps of C2 are as follows: C2-1: Direct communication between virtio-GPU front-end and back-end: Used for direct communication between virtio-GPU front-end and GPU back-end in SOS, eliminating encoder conversion and interacting efficiently through standard kernel interfaces; C2-2: Sending of data stream addresses and processing instructions: used by the virtio-GPU front end to send data stream addresses and processing instructions to the GPU back end in SOS, directly locate data and specify operations, avoid data copying, and improve the performance bottleneck when processing large data images simultaneously using the standard graphics library interface.

5. The method for sharing and using a multi-virtual machine GPU sharing system according to claim 2, characterized in that: The specific steps of C3 are as follows: C3-1:Virtio-GPU access mechanism, including: Control flow: used to process the exchange of commands and status information; Data flow: the virtual machine directly accesses the GPU data on the host machine without copying the data. C3-2: 3D engine library and GPU underlying interface: used to directly access the GPU underlying interface and control the rendering and computing capabilities of the GPU.

6. The method for sharing and using a multi-virtual machine GPU sharing system according to claim 5, characterized in that: The specific steps of C3 are as follows: C3-3: Virtual-OpenGL and GPU Encoder: The GPU encoder is used to efficiently encode and serialize the API calls of the 3D engine, reducing the memory usage and complexity in the user operating system UOS. These serialized data are transmitted to the GPU backend in the service operating system SOS through vhost-dmabuf or network mechanism for decoding and execution.

7. The method for sharing and using a multi-virtual machine GPU sharing system according to claim 6, characterized in that: The specific steps of C3 are as follows: C3-4: ASG and fast IPC: ASG is a communication method that implements fast inter-process communication IPC through an acceleration channel and is used in scenarios that require low latency and high bandwidth.

8. The secure sharing method of a multi-virtual machine GPU sharing system according to claim 1, characterized in that: The first step includes the following: D1: Separation of server virtual machines and user virtual machines: In the T-Hyper virtualization architecture, the server virtual machine SOS and the user virtual machine UOS are clearly separated in terms of GPU resource access, achieving efficient resource sharing and secure isolation.

9. The method for secure GPU sharing for multiple virtual machines according to claim 8, characterized in that: The second step includes the following: D2: Fine-grained access control: The microkernel-based T-Hyper divides system resources into fine-grained categories using a capability-based approach.

10. The GPU secure sharing method for multiple virtual machines according to claim 9, characterized in that: The third step includes the following: D3: GPU address protection: To prevent potential DMA attacks, GPU addresses are protected by the SMMU system memory management unit or MPU memory protection unit hardware isolation mechanism.

Citation Information

Patent Citations

  • A GPU service redirection system and method based on cloud and terminal GPU fusion

    CN109582425A

  • Method and apparatus for efficient use of graphics processing resources in a virtualized execution environment

    CN109643291A

  • GPU virtualization implementation method based on SR-IOV technology

    CN115904617A

  • Heterogeneous intelligent computing platform virtualization management system and method

    CN117707693A

  • Apparatus and method for dynamic provisioning, quality of service, and scheduling

    US20180293185A1

Cited By

  • High-reliability multiple redundancy virtualization method based on microkernel

    CN119883516A

  • A high-reliability multi-redundancy virtualization method based on a microkernel

    CN119883516B