A multi-virtual machine-oriented GPU security sharing method

Abstract

The application relates to the technical field of interface management, and discloses a GPU (Graphics Processing Unit) security sharing method for multiple virtual machines, which solves the problem that existing virtual machines or physical machines are usually allocated independent GPU resources, leading to difficulty in efficient utilization of the GPU resources. Through an intermediate layer image engine library, a virtual image engine interface, an upper layer GPU encoder and virtio-GPU front and rear end components, indirect access of a UOS (Unified Operating System) application program to GPU resources is realized, the access efficiency and performance are improved, and the security and flexibility of the GPU resources are ensured. Through GPU virtualization, multiple virtual GPU instances are created on a physical GPU, the physical GPU resources are divided into multiple virtual slices, and each virtual slice is allocated to a virtual machine. This mode not only improves the utilization rate of the GPU resources, but also reduces the management cost.

Description

Technical Field

[0001] This invention relates to the field of interface management, specifically to a method for secure GPU sharing across multiple virtual machines. Background Technology

[0002] With the rapid development of technology, GPUs, as powerful accelerator chips, have been widely used in high-performance computing, artificial intelligence, machine learning, and big data processing. The parallel computing capabilities of GPUs have given them a huge advantage in complex data processing and algorithm acceleration. However, in practical applications, the effective management and secure sharing of GPU resources has become an urgent problem to be solved.

[0003] In the traditional approach, each virtual machine or physical machine is typically allocated independent GPU resources, making it difficult to utilize GPU resources efficiently. Especially during off-peak hours, GPU resources are often idle, resulting in resource waste. GPU devices themselves are expensive, and for scenarios requiring a large amount of computing resources, the purchase and maintenance costs are very high. In an environment where multiple virtual machines share GPU resources, there are serious security risks. Malicious virtual machines can steal data from other virtual machines or illegally occupy too many GPU resources, causing other virtual machines to experience performance degradation or even crash.

[0004] To address the aforementioned issues, an innovative design was developed based on existing methods for secure GPU sharing across multiple virtual machines. Summary of the Invention

[0005] The purpose of this invention is to provide a secure GPU sharing method for multiple virtual machines. By using this device, the problem of existing virtual machines or physical machines being allocated independent GPU resources, which makes it difficult to efficiently utilize GPU resources, is solved.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for secure GPU sharing across multiple virtual machines, comprising:

[0007] T-Hyper: T-Hyper is a trusted virtual machine management system responsible for creating management server virtual machines and user virtual machines, ensuring the secure isolation of virtual machines and the reasonable allocation of resources;

[0008] The upper-layer GPU backend of the management server virtual machine is responsible for managing and accessing GPU resources, optimizing the execution of GPU tasks through the middle-layer graphics engine library, and efficiently accessing shared memory using DMA direct memory access technology.

[0009] Virtual Graphics Engine Interface: Provides a set of upper-layer library modules as a bridge for UOS user operating system applications to access GPU graphics processing functions. Through standard interface encapsulation, it reduces the application's dependence on the underlying GPU hardware and provides UOS applications with an interface to access upper-layer graphics processing functions.

[0010] Upper-layer GPU encoder: Used to encode the data generated by the virtual graphics engine interface, convert it into a format suitable for network transmission, and send it to the upper-layer GPU backend of the management server virtual machine for processing;

[0011] Virtual GPU upper-layer interface: responsible for directly providing UOS applications with an interface to access the underlying graphics processing functions, bypassing the traditional graphics stack, improving the efficiency and performance of artwork access;

[0012] The virtio-GPU front end is responsible for calling the intermediate layer graphics engine library and accessing shared memory through DMA technology. It works with the virtio-GPU back end to realize the virtualization of GPU resources and efficient communication and data transfer between SOS and GPU resources, supporting flexible allocation and sharing of GPU resources.

[0013] Furthermore, the method includes the following three approaches:

[0014] C1: Access GPU hardware through the basic graphics access interface;

[0015] C2: Communicates and accesses the GPU backend in SOS via the virtio-GPU frontend;

[0016] C3: Directly use GPU resources by calling methods of the underlying GPU interface through the upper-level interface of the virtual GPU.

[0017] Furthermore, the specific steps of C1 are as follows:

[0018] C1-1: The UOS application calls the virtual image engine interface, which is responsible for calling the upper-layer GPU encoder to encode the interface according to the specified rules and then pass it to the upper-layer GPU backend through a fast channel.

[0019] C1-2: The upper-layer GPU backend is responsible for receiving and decoding encoded requests, identifying the actual image engine library and interface name, calling the corresponding image engine library according to the request, and executing the actual GPU operation. It can support multiple image engine libraries and realize flexible scheduling and sharing of GPU resources.

[0020] Furthermore, the specific steps of C2 are as follows:

[0021] C2-1: Direct communication between the virtio-GPU front-end and back-end: Used for virtio-GPU to achieve direct communication between the front-end and the GPU back-end in SOS, eliminating encoder conversion and enabling efficient interaction through the standard kernel interface;

[0022] C2-2: Sending Data Stream Address and Processing Instructions: Used by the virtio-GPU front end to send data stream addresses and processing instructions to the GPU back end in SOS, directly locating the data and specifying the operation, avoiding data copying, and improving the performance bottleneck when processing large data images simultaneously using standard graphics library interfaces.

[0023] Furthermore, the specific steps of C3 are as follows:

[0024] C3-1: Virtual-GPU access mechanism, including: Control flow: used to handle the exchange of commands and status information; Data flow: the virtual machine directly accesses GPU data on the host machine without copying the data.

[0025] C3-2: 3D Engine Library and GPU Low-Level Interface: Used to directly access the GPU's low-level interface and control the GPU's rendering and computing capabilities.

[0026] Furthermore, the specific steps of C3 are as follows:

[0027] C3-3: Virtual-OpenGL and GPU Encoder: The GPU encoder efficiently encodes and serializes the API calls of the 3D engine, reducing memory usage and complexity in the user operating system (UOS). The serialized data is then transmitted to the GPU backend in the service operating system (SOS) via vhost-dmabuf or a network mechanism for decoding and execution.

[0028] Furthermore, the specific steps of C3 are as follows:

[0029] C3-4: ASG and Fast IPC: ASG is a communication method that enables fast inter-process communication (IPC) through an accelerated channel, suitable for scenarios requiring low latency and high bandwidth.

[0030] Furthermore, the first step includes the following:

[0031] D1: Separation of Server Virtual Machines and User Virtual Machines: In the T-Hyper virtualization architecture, a clear separation of the server virtual machine (SOS) and the user virtual machine (UOS) in GPU resource access is achieved, realizing efficient resource sharing and secure isolation.

[0032] Furthermore, the process includes the following second step:

[0033] D2: Fine-grained access control: T-Hyper, based on a microkernel, performs fine-grained partitioning of system resources through a capability-based approach.

[0034] Furthermore, the process includes the following third step:

[0035] D3: GPU Address Protection: To prevent potential DMA attacks, GPU addresses are protected by hardware isolation mechanisms through the SMMU (System Memory Management Unit) or MPU (Memory Protection Unit).

[0036] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0037] This invention proposes a secure GPU sharing method for multiple virtual machines. Existing virtual machines or physical machines are typically allocated independent GPU resources, making it difficult to utilize GPU resources efficiently. This invention, however, enables UOS applications to indirectly access GPU resources through components such as a middle-layer image engine library, a virtual image engine interface, an upper-layer GPU encoder, and the virtualo-GPU front-end and back-end. This improves access efficiency and performance while ensuring the security and flexibility of GPU resources. By creating multiple virtual GPU instances on the physical GPU through GPU virtualization, the physical GPU resources are divided into multiple virtual slices, each of which is assigned to a virtual machine. This approach not only improves GPU resource utilization but also reduces management costs. Attached Figure Description

[0038] Figure 1 This is an overall diagram of the multi-virtual machine GPU sharing system method of the present invention;

[0039] Figure 2 This is a flowchart of the process of accessing GPU hardware using the basic graphics access interface according to the present invention.

[0040] Figure 3 This is a flowchart illustrating the process of accessing GPU resources via the virtio-GPU frontend using the basic image access interface, as described in this invention.

[0041] Figure 4 The flowchart of the present invention is shown below, illustrating the process of directly calling GPU resources using the upper-layer interface of the virtual GPU. Detailed Implementation

[0042] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] To further understand the content of this invention, a detailed description of the invention will be provided in conjunction with the accompanying drawings.

[0044] Please see Figure 1 A method for secure GPU sharing across multiple virtual machines, comprising:

[0045] T-Hyper: T-Hyper is a trusted virtual machine management system responsible for creating management server virtual machines and user virtual machines, ensuring the secure isolation of virtual machines and the reasonable allocation of resources;

[0046] The upper-layer GPU backend of the management server virtual machine is responsible for managing and accessing GPU resources, optimizing the execution of GPU tasks through the middle-layer graphics engine library, and efficiently accessing shared memory using DMA direct memory access technology.

[0047] Virtual Graphics Engine Interface: Provides a set of upper-layer library modules as a bridge for UOS user operating system applications to access GPU graphics processing functions. Through standard interface encapsulation, it reduces the application's dependence on the underlying GPU hardware and provides UOS applications with an interface to access upper-layer graphics processing functions.

[0048] Upper-layer GPU encoder: Used to encode the data generated by the virtual graphics engine interface, convert it into a format suitable for network transmission, and send it to the upper-layer GPU backend of the management server virtual machine for processing;

[0049] Virtual GPU upper-layer interface: responsible for directly providing UOS applications with an interface to access the underlying graphics processing functions, bypassing the traditional graphics stack, improving the efficiency and performance of artwork access;

[0050] The virtio-GPU front end is responsible for calling the intermediate layer graphics engine library and accessing shared memory through DMA technology. It works with the virtio-GPU back end to realize the virtualization of GPU resources and efficient communication and data transfer between SOS and GPU resources, supporting flexible allocation and sharing of GPU resources.

[0051] The present invention will be further described below with reference to embodiments.

[0052] Please see Figure 1The shared usage method of a multi-virtual machine GPU sharing system, as described above, includes the following steps: T-Hyper is a trusted virtual machine management system responsible for creating management server virtual machines and user virtual machines, ensuring secure isolation of virtual machines and reasonable allocation of resources; the upper-layer GPU backend of the management server virtual machine is responsible for managing and calling GPU resources, optimizing GPU task execution through a middle-layer graphics engine library, and efficiently accessing shared memory using DMA direct memory access technology; the virtual graphics engine interface provides a set of upper-layer library modules, serving as a bridge for UOS user operating system applications to access GPU graphics processing functions. Through standard interface encapsulation, it reduces the application's dependence on the underlying GPU hardware and provides UOS applications with access to upper-layer graphics processing functions. Access Interface; Upper-layer GPU Encoder: Used to encode and process data generated by the virtual graphics engine interface, converting it into a format suitable for network transmission, and sending it to the upper-layer GPU backend of the management server virtual machine for processing; Virtual GPU Upper-layer Interface: Responsible for directly providing the UOS application with an interface to access the underlying graphics processing functions, bypassing the traditional graphics stack, improving artwork access efficiency and performance; virtio-GPU Frontend: Responsible for calling the middle-layer graphics engine library and accessing shared memory through DMA technology to cooperate with the virtio-GPU backend to realize the virtualization of GPU resources and efficient communication and data transmission between SOS and GPU resources, supporting flexible allocation and sharing of GPU resources, and improving graphics processing efficiency and data encoding efficiency.

[0053] Specifically, firstly, the T-Hyper virtual machine management system is installed and configured on the physical hardware. As the core of the entire virtualization environment, T-Hyper is responsible for creating and managing one server virtual machine (SOS) and multiple user virtual machines (UOS), ensuring that each virtual machine has independent resource allocation and security isolation. Then, GPU drivers and intermediate-layer graphics engine libraries are deployed in the SOS. The intermediate-layer graphics engine library is responsible for optimizing GPU task execution, improving graphics processing efficiency, and utilizing DMA (Direct Memory Access) technology to efficiently access and manage shared memory regions, enabling fast data transfer between the SOS and UOS. Next, a virtual graphics engine interface component is deployed in the UOS, acting as a bridge for UOS applications to access GPU graphics processing functions. The virtual graphics engine interface is encapsulated through a standard interface, reducing the application's dependence on the underlying GPU hardware. This allows the upper-layer GPU encoder to encode the data generated by the virtual graphics engine interface, converting it into a format suitable for network transmission, and sending it to the upper-layer GPU backend in the SOS for processing. Thus, the virtual GPU upper-layer interface directly provides the UOS application with an interface to access the underlying graphics processing functions, bypassing the traditional graphics stack and improving access efficiency and performance. Finally, a virtual GPU interface is deployed in the SOS. The virtual-GPU front-end serves as an efficient communication and data transfer interface between the SOS and GPU resources. It calls the intermediate-layer graphics engine library and uses DMA technology to access shared memory, working in conjunction with the virtual-GPU back-end to virtualize GPU resources and flexibly allocate and share them between the SOS and GPU. Through T-Hyper's fine-grained management capabilities, it ensures direct access to GPU hardware by the SOS while restricting direct access by the UOS. Utilizing the virtual graphics engine interface and the upper-layer GPU encoder, it enables indirect access to GPU resources by UOS applications, ensuring security. Through the collaboration between the virtual-GPU front-end and back-end, it achieves flexible allocation and efficient sharing of GPU resources between the SOS and multiple UOS systems. Finally, it continuously optimizes the algorithms of the intermediate-layer graphics engine library and the upper-layer GPU encoder to improve graphics processing and data encoding efficiency. It monitors and manages GPU resource usage, dynamically adjusting resource allocation based on the actual needs of the virtual machine to ensure overall system performance stability and optimization.

[0054] Please see Figure 2The UOS application accesses the GPU hardware through the basic graphics access interface; the UOS application calls the virtual image engine interface, which is responsible for calling the upper-layer GPU encoder to encode the interface according to the specified rules and pass it to the upper-layer GPU backend through a fast channel; the upper-layer GPU backend is responsible for receiving the encoded request and decoding it, identifying the real image engine library and interface name, calling the corresponding image engine library according to the request, and executing the actual GPU operation. It can support multiple image engine libraries, realize flexible scheduling and sharing of GPU resources.

[0055] Specifically, the process begins with a request initiated by the UOS application. When a user or application within the UOS system needs to perform graphics processing or computational tasks, it invokes the virtual graphics engine interface. These interfaces are a standardized set of APIs provided by the system for upper-layer applications, encapsulating access to GPU hardware. Next, the virtual graphics engine interface processes the request. Upon receiving the request from the UOS application, the virtual graphics engine interface forwards it to the upper-layer GPU encoder based on the request type (e.g., OpenGL rendering, CUDA computation) and parameters. The upper-layer GPU encoder then encodes the request according to predetermined encoding rules. This encoding process involves converting the interface name, parameters, and other information in the request into a universal, easily transmittable format. The encoded data contains sufficient information for the upper-layer GPU backend to decode and identify the specific operational requirements. Finally, the request is transmitted via a fast IPC channel. The encoded request data is passed to the upper-layer GPU backend through a shared-memory-based fast IPC (Inter-Process Communication) channel. This channel design ensures... High-efficiency and low-latency data transmission are key to achieving high-performance GPU access. Further, the upper-layer GPU backend decodes and executes the data. After receiving the encoded request data, the upper-layer GPU backend performs decoding to identify the specific graphics engine library (such as OpenGL, CUDA, etc.) and interface name. Then, based on the request content, it calls the corresponding graphics engine library to execute the actual GPU operation. This process supports multiple graphics engine libraries, enabling flexible scheduling and sharing of GPU resources. Finally, the processing result is returned. After the GPU operation is completed, the processing result is returned to the UOS application via the reverse path. If the operation is successful, the application can continue to execute subsequent tasks; if it fails, it is handled accordingly based on the error message. Through this process, applications in the UOS system can easily access GPU hardware using basic graphics access interfaces (such as CUDA, OpenGL, etc.) without directly concerning themselves with the physical GPU driver and low-level details. This mechanism not only improves system compatibility and portability but also enhances system performance by optimizing the call path and reducing intermediate layer overhead.

[0056] Please see Figure 3 It communicates with the GPU backend in SOS through the virtio-GPU frontend. Direct communication between the virtio-GPU frontend and backend: This enables direct communication between the virtio-GPU frontend and the GPU backend in SOS, eliminating encoder conversion and enabling efficient interaction through the standard kernel interface. Sending data stream addresses and processing instructions: This allows the virtio-GPU frontend to send data stream addresses and processing instructions to the GPU backend in SOS, directly locating data and specifying operations, avoiding data copying, and improving the performance bottleneck when processing large amounts of image data simultaneously using the standard graphics library interface.

[0057] Specifically, firstly, the UOS application initiates a request. When an application in the UOS user virtual machine needs to perform a graphics processing task, it sends a request to the virtual io-GPU frontend through basic image access interfaces (such as OpenGL, CUDA, etc.). These requests contain the specific instructions for graphics processing and the required data stream addresses. Secondly, after receiving the request from the UOS application, the virtual graphics engine interface forwards the request to the virtual io-GPU frontend according to the request type (such as OpenGL rendering, CUDA computation, etc.) and parameters. Upon receiving the request, the virtual io-GPU frontend does not perform complex encoding and decoding operations, but directly packages the data stream addresses and processing instructions in the request into a data packet format specified by the virtual io protocol. These data packets contain all the information required for GPU operation. Then, the virtual io-GPU frontend uses standard virtual io protocols to process the data. The rtio interface sends data packets to the GPU backend in SOS on the host machine. Upon receiving the data packets, the backend parses the data stream address and processing instructions. It utilizes DMA (Direct Memory Access) technology to directly read the corresponding image data from the host machine's physical memory and transfer this data to the physical GPU hardware for processing. DMA technology reduces CPU intervention and improves data transfer efficiency. Through this process, the UOS user virtual machine can efficiently interact with the GPU resources on the host machine using the vi rtio-GPU frontend, thereby achieving fast access and efficient utilization of GPU resources. This not only improves system compatibility and portability but also enhances system performance by reducing overhead during data transfer and processing.

[0058] Please see Figure 4Virtual GPU directly accesses GPU resources by calling methods of the underlying GPU interface through the upper-level interface of Virtual GPU. Virtual-GPU access mechanisms include control flow for exchanging commands and status information, and data flow for the virtual machine to directly access GPU data on the host machine without data copying. The 3D engine library and GPU underlying interface directly access the GPU underlying interface and control the GPU's rendering and computing capabilities. Virtual-OpenGL and GPU encoder efficiently encode and serialize the 3D engine's API calls, reducing memory usage and complexity in the user operating system (UOS). This serialized data is transmitted to the GPU backend in the service operating system (SOS) via vhost-dmabuf or network mechanisms for decoding and execution. ASG and fast IPC: ASG is a communication method that achieves fast inter-process communication (IPC) through an accelerated channel, improving data transmission and processing efficiency for scenarios requiring low latency and high bandwidth.

[0059] Specifically, firstly, the UOS application initiates a request. When an application in the UOS user virtual machine needs to perform graphics processing tasks, it sends a request to the Virtio-GPU frontend through the upper-layer interface of the Virtual GPU. These requests contain the specific graphics processing instructions and the required data stream addresses. The user operating system UOS can directly send GPU instructions to the GPU hardware in the host operating system SOS without going through the intermediary of the graphics processing library. UOS communicates with the GPU backend in SOS through the standard Virtio interface, and then Virtio... The io-GPU protocol divides communication into control flow and data flow. The control flow is used to transmit GPU commands and status information, while the data flow is used to efficiently transmit the large amounts of data required for GPU processing. In scenarios requiring low latency and high bandwidth, ASG (Accelerated Service Gateway) technology is used to establish a fast IPC (Inter-Process Communication) channel. GPU commands and status information are transmitted quickly between UOS and SOS through the ASG channel, reducing transmission latency and bandwidth consumption. In data flow transmission, ION technology is used to create a shared memory region, enabling UOS and SOS to directly access the same physical memory, thereby avoiding data copying and additional memory overhead, reducing the overhead of the intermediate layer, and improving the efficiency of data transmission and processing, thus meeting the needs of high-performance computing and graphics processing applications.

[0060] The aforementioned method for secure GPU sharing across multiple virtual machines includes the following steps: Separation of server virtual machines and user virtual machines: In the T-Hyper virtualization architecture, a clear separation of server virtual machines (SOS) and user virtual machines (UOS) in GPU resource access is achieved, realizing efficient resource sharing and secure isolation; Fine-grained access control: Based on a microkernel, T-Hyper performs fine-grained partitioning of system resources through a capability-based approach; GPU address protection: To prevent potential DMA attacks, GPU addresses are protected through hardware isolation mechanisms using the SMMU (System Memory Management Unit) or MPU (Memory Protection Unit).

[0061] Specifically, the implementation of the GPU secure sharing method involves building a virtualization architecture using the microkernel-based HyperVirus as the virtualization platform. This platform supports the creation of multiple virtual machines, including a single server virtual machine (SOS) and multiple user virtual machines (UOS). GPU drivers are installed and configured in the SOS to ensure that the SOS can directly access GPU hardware addresses. Secondly, GPU access permissions are controlled through HyperVirus's fine-grained management capabilities, strictly limiting GPU access to the SOS. The UOS cannot directly access the GPU hardware addresses and can only do so through the virtualization interface Virtual... The virtualization-GPU communicates with the GPU backend in the SOS. Security policies for GPU access are configured in the SOS, including restricting different applications' access to the GPU interface. Through capability-level segmentation, application access permissions to different GPU access channels can be restricted, as well as direct access and manipulation of the graphics engine library through low-level interfaces. SMMU or MPU chip isolation mechanisms protect GPU addresses, preventing DMA attacks and ensuring the security and integrity of GPU memory. SMMU / MPU is configured to restrict access to the GPU address space, ensuring that only authorized processes or devices can access GPU memory. This is further implemented through a virtualization interface. The Virtualization-GPU backend is implemented in the SOS, handling GPU access requests from the UOS. The Virtualization-GPU backend communicates with the Virtualization-GPU frontend in the UOS via control and data flows. Control flow handles command and status information exchange, while data flow allows the virtual machine to directly access GPU data on the host machine through a shared memory mechanism, eliminating the need for data copying. The o-GPU front-end runs in UOS, providing a virtualized GPU access interface for applications in UOS. Applications send GPU commands and data to the Virtual o-GPU back-end in SOS for processing, thereby optimizing performance and scheduling resources. A GPU scheduler is implemented in SOS to manage and schedule the use of GPU resources. The scheduler dynamically allocates GPU resources to each virtual machine based on the needs of the virtual machine and the available GPU resources. By optimizing the virtualization interface and GPU scheduling algorithm, the utilization rate and performance of GPU resources are improved. A GPU encoder can be used to efficiently encode and serialize the API calls of the 3D engine, reducing memory consumption and complexity.Simultaneously, technologies such as ASG (Accelerated Service Gateway) are utilized to achieve rapid inter-process communication (IPC), reducing transmission latency and bandwidth consumption. Secure GPU sharing can be achieved based on the microkernel-based Hypervisor, ensuring that the stability and security of SOS are not affected by potential risks in UOS. Furthermore, fine-grained management and optimization techniques improve GPU resource utilization and performance.

[0062] In traditional GPU virtualization, GPUs are typically allocated in slices, with time slices allocated to virtual machines. Some GPU scheduling algorithms use priority-based methods, setting priorities for each task and scheduling higher-priority tasks first. However, these settings are generally static and cannot dynamically adjust GPU resource usage based on GPU load and performance. Therefore, a secure sharing approach is needed to ensure clear access control and isolation mechanisms for each virtual machine when accessing GPU resources. GPU resources are dynamically allocated based on the real-time needs of the virtual machines and system load. Dynamic scheduling of GPU resources in the virtualization environment includes a GPU resource manager, a Virtual Machine Monitor (VMM), and a credit point management module. The GPU resource manager receives GPU resource requests from virtual machines and allocates resources based on weights, capabilities, and credit points. The VMM monitors the running status and GPU resource usage of each virtual machine, providing necessary information to the GPU resource manager. The credit point management module dynamically adjusts the credit points of each task based on task execution and preset rules, setting task priorities accordingly. A weight and capability value are assigned to each virtual machine during creation or configuration.The weight value determines the relative priority and resource allocation ratio of virtual machines when sharing GPU resources. For virtual machine A with a weight of 3 and virtual machine B with a weight of 1, when they share a physical GPU, the GPU resource manager will allocate time slices to these two virtual machines in a 3:1 ratio. The capability value limits the maximum frequency of GPU usage by a virtual machine per unit time. Even without competition from other virtual machines, a virtual machine with a lower capability value will not obtain GPU resources exceeding its capability limit. Based on the credit point priority mechanism, an initial credit point is assigned to each task, and different credit point consumption per unit time is set according to the task's priority and execution requirements. When a task is executed, its credit points will be dynamically consumed according to the rate of credit point consumption per unit time. Based on the remaining credit points of the task, the credit point management module divides the task priority into four levels: high, medium, low, and idle. The higher the priority of the task, the less likely its GPU resources are to be preempted by other tasks. When a high-priority task needs GPU resources, the GPU resource manager will prioritize filling them. To meet the needs, even if it means interrupting currently executing low-priority tasks, the system dynamically schedules and optimizes resources, monitors GPU resource usage and task execution status in real time, and dynamically increases the weight and capability of relevant virtual machines for high-response tasks in the vehicle system, such as image collection and processing for automatic parking, ensuring they can obtain sufficient GPU resources first. Based on task execution and system load, the system dynamically adjusts credit point allocation and priority settings to optimize GPU resource utilization and system performance. Through the collaborative work of the virtual machine monitor and GPU resource manager, the system ensures secure isolation between virtual machines and reasonable allocation of GPU resources, monitors and prevents illegal resource occupation and malicious attacks between virtual machines, and guarantees system stability and security. Through the above specific implementation methods, the virtualization-based GPU resource dynamic scheduling system proposed in this invention can effectively achieve efficient, flexible, and secure dynamic scheduling of GPU resources, meeting the GPU resource needs of different virtual machines and application scenarios.

[0063] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0064] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A multi-virtual machine GPU sharing system, characterized in that: include: T-Hyper: T-Hyper is a trusted virtual machine management system responsible for creating management server virtual machines (SOS) and user virtual machines (UOS), ensuring the secure isolation of virtual machines and the reasonable allocation of resources; Upper-layer GPU backend: The upper-layer GPU backend of the management server virtual machine SOS optimizes the execution of GPU tasks through the middle-layer image engine library and efficiently accesses shared memory using DMA direct memory access technology; Virtual Graphics Engine Interface: Provides a set of upper-level library modules as a bridge for user virtual machine (UOS) applications to access GPU graphics processing functions. Through standard interface encapsulation, it reduces the application's dependence on the underlying GPU hardware and provides the user virtual machine (UOS) application with an interface to access upper-level graphics processing functions. Upper-layer GPU encoder: Used to encode the data generated by the virtual graphics engine interface, convert it into a format suitable for network transmission, and send it to the upper-layer GPU backend of the management server virtual machine SOS for processing; Virtual GPU upper-layer interface: responsible for directly providing the user virtual machine UOS application with an interface to access the underlying graphics processing functions, bypassing the traditional graphics stack and improving access efficiency and performance; virtio-GPU front end: Responsible for calling the middle-layer graphics engine library and accessing shared memory through DMA technology to cooperate with the upper-layer GPU back end to realize the virtualization of GPU resources and efficient communication and data transmission between the server virtual machine SOS and GPU resources, supporting flexible allocation and sharing of GPU resources.

2. The method for sharing a multi-virtual machine GPU sharing system as described in claim 1, characterized in that: Including the following three methods: C1: Access GPU hardware through the basic graphics access interface; C2: Communicates and accesses the upper-layer GPU backend in the server virtual machine SOS through the virtio-GPU frontend; C3: Directly use GPU resources by calling methods of the underlying GPU interface through the upper-level interface of virtualGPU.

3. The method for sharing a multi-virtual machine GPU sharing system as described in claim 2, characterized in that: The specific steps of C1 are as follows: C1-1: The user virtual machine UOS application calls the virtual image engine interface, which is responsible for calling the upper-layer GPU encoder to encode the interface according to the specified rules and then pass it to the upper-layer GPU backend through the fast channel. C1-2: The upper-layer GPU backend is responsible for receiving and decoding encoded requests, identifying the actual image engine library and interface name, calling the corresponding image engine library according to the request, and executing the actual GPU operation. It can support multiple image engine libraries and realize flexible scheduling and sharing of GPU resources.

4. The method for sharing a multi-virtual machine GPU sharing system as described in claim 2, characterized in that: The specific steps of C2 are as follows: C2-1: Direct communication between the virtio-GPU front-end and the upper-layer GPU back-end: Used for virtio-GPU to achieve direct communication between the front-end and the GPU back-end in the server virtual machine SOS, eliminating encoder conversion and enabling efficient interaction through the standard kernel interface; C2-2: Sending Data Stream Address and Processing Instructions: Used by the virtio-GPU front end to send data stream addresses and processing instructions to the upper-layer GPU back end in the server virtual machine SOS, directly locating the data and specifying the operation, avoiding data copying, and improving the performance bottleneck when processing large data images while using standard graphics library interfaces.

5. The method for sharing a multi-virtual machine GPU sharing system as described in claim 2, characterized in that: The specific steps of C3 are as follows: C3-1: Virtio-GPU access mechanism, including: Control flow: used to handle the exchange of commands and status information; Data flow: the virtual machine directly accesses GPU data on the host machine without copying the data; C3-2: 3D Engine Library and GPU Low-Level Interface: Used to directly access the GPU's low-level interface and control the GPU's rendering and computing capabilities.

6. The method for sharing a multi-virtual machine GPU sharing system according to claim 5, characterized in that: The specific steps of C3 are as follows: C3-3: Virtual-OpenGL and GPU Encoder: The GPU encoder efficiently encodes and serializes the API calls of the 3D engine, reducing memory usage and complexity in the user virtual machine (UOS). This serialized data is transmitted to the upper-level GPU backend in the server virtual machine (SOS) via vhost-dmabuf or network mechanisms for decoding and execution.

7. The method for sharing a multi-virtual machine GPU sharing system according to claim 6, characterized in that: The specific steps of C3 are as follows: C3-4: ASG and Fast IPC: ASG is a communication method that enables fast inter-process communication (IPC) through an accelerated channel, suitable for scenarios requiring low latency and high bandwidth.

8. The secure sharing method for a multi-virtual machine GPU sharing system as described in claim 1, characterized in that: The first step includes the following: D1: Separation of Server Virtual Machines and User Virtual Machines: In the T-Hyper virtualization architecture, a clear separation of the server virtual machine (SOS) and the user virtual machine (UOS) in GPU resource access is achieved, realizing efficient resource sharing and secure isolation.

9. A secure sharing method for a multi-virtual machine GPU sharing system according to claim 8, characterized in that: This includes the following second step: D2: Fine-grained access control: T-Hyper, based on a microkernel, performs fine-grained partitioning of system resources through a capability-based approach.

10. A secure sharing method for a multi-virtual machine GPU sharing system according to claim 9, characterized in that: This includes the following third step: D3: GPU Address Protection: To prevent potential DMA attacks, GPU addresses are protected by hardware isolation mechanisms through the SMMU (System Memory Management Unit) or MPU (Memory Protection Unit).

Images