Image rendering method based on SPICE protocol and virtual GPU

Abstract

The invention relates to the technical field of computer virtualization, and discloses an image rendering method based on an SPICE protocol and a virtual GPU, and the method comprises the steps: enabling virtualization configuration on a physical computer node, installing GPU equipment and a driver thereof, creating a virtual machine on the node, and distributing the virtual GPU; installing a display drive model in the virtual machine to redirect the drive, intercepting and redirecting the rendering and computing tasks to the virtual GPU for processing; and installing the modified QXL driver, identifying the QXL driver as a PCI device driver, capturing a picture rendered by the virtual GPU, and transmitting graphic output through an SPICE protocol. According to the method, by combining the virtual GPU and the SPICE protocol, efficient graphic rendering and remote display are achieved, the problem that the performance of a traditional virtual graphics card is limited is solved, and the method is suitable for virtual machine application scenes needing high-performance graphic processing.

Description

Technical Field

[0001] The present application relates to the field of computer virtualization technology, for example, to an image rendering method based on SPICE protocol and virtual GPU. Background Art

[0002] In current virtualization technologies, users often use QEMU / KVM and SPICE to implement virtualization and remote display technologies. Among them, remote display using SPICE mainly relies on the virtual graphics card QXL, which is low-cost, easy to install and configure, and has good compatibility. However, the rendering capability of the QXL graphics card is limited by software-level simulation and cannot give full play to the efficient computing power of the hardware GPU.

[0003] This limitation is particularly evident when processing complex 3D graphics and high-resolution videos, resulting in degraded image quality and poor user experience. To solve this problem, the industry currently has solutions such as GPU direct passthrough and virtual GPU, but when using virtual GPU for rendering, the QXL device loses its graphics card capabilities, which causes the SPICE protocol to lack image input and cannot achieve remote display. In addition, the combination of traditional virtual machine graphics rendering and SPICE protocol has performance bottlenecks and is difficult to meet the needs of high-performance graphics applications.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present application, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] In order to provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. The summary is not an extensive review, nor is it intended to identify key / critical elements or delineate the scope of protection of these embodiments, but rather serves as a prelude to the detailed description that follows.

[0006] The present disclosure provides an image rendering method based on the SPICE protocol and a virtual GPU, the method comprising: At least one physical computer is used as a node; virtualization configuration is enabled on the physical computer node, and at least one GPU device is installed; a GPU driver and a GPU virtualization driver are installed on the node, wherein the GPU driver is used to ensure the normal operation of the GPU, and the GPU virtualization driver is used to provide a virtual GPU for the virtual machine; at least one virtual machine is created on the node, and a virtual GPU is allocated to it; a virtual GPU driver is installed in the virtual machine to ensure that the virtual GPU can perform rendering tasks normally; Installing a display driver model redirection driver in the virtual machine, wherein the redirection driver is used to intercept and redirect the rendering task of the virtual computer to the virtual GPU for processing; Install the modified QXL driver in the virtual machine. The QXL driver will be recognized by the system as a PCI device driver to capture the virtual GPU rendering screen and transmit the virtual machine's graphics output to the SPICE server through the SPICE protocol. The images rendered by the virtual GPU are transmitted to the SPICE client via the SPICE protocol for remote display.

[0007] In some embodiments, the redirection driver is used to intercept and redirect the rendering task of the virtual computer to the virtual GPU for processing, including: The redirection driver intercepts all DirectX API calls sent from the operating system to the GPU by replacing the function pointer in the virtual function table of the DirectX object in the original function with a hook function. When the original function is called, the hook function is executed to intercept the rendering task.

[0008] In some embodiments, the method comprises: When the hook function in the redirection driver intercepts a DirectX API call, it obtains the parameters of the API call, wherein the parameters of the API call describe a specific rendering task.

[0009] In some embodiments, the method further comprises: The hook function in the redirection driver encapsulates the parameters of the API call into a task object that can be recognized by the virtual GPU, including setting the task type, rendering target, vertex data and texture.

[0010] In some embodiments, the method further comprises: The hook function in the redirection driver calls the interface of the virtual GPU driver and adds the encapsulated task object to the task queue of the virtual GPU; After receiving the task, the virtual GPU executes the rendering task through its driver, and these tasks are executed on the hardware of the virtual GPU; After the virtual GPU completes the task, it generates the corresponding graphics output and passes it to the QXL driver through the shared memory mechanism.

[0011] In some embodiments, the modified QXL driver is identified as a PCI device driver by the system by defining a PCI device identifier, where the PCI device identifier includes a Vendor ID and a Device ID registered in the operating system.

[0012] In some embodiments, the modified QXL driver implements a probe function to handle device initialization, including enabling the device, requesting device resources, and mapping device memory space.

[0013] In some embodiments, the modified QXL driver defines a shared memory area to achieve data transmission between the virtual GPU driver and the QXL driver.

[0014] In some embodiments, the modified QXL driver defines an interface for obtaining graphics output to call the API of the virtual GPU driver to transfer data from the virtual GPU to the memory space managed by the QXL driver.

[0015] In some embodiments, the modified QXL driver compresses the graphic data using the H.264 compression algorithm and transmits the compressed graphic data to the SPICE server via the SPICE protocol.

[0016] The image rendering method based on the SPICE protocol and virtual GPU provided in the embodiments of the present disclosure can achieve the following technical effects: The present disclosure realizes efficient graphics rendering and remote display by combining virtual GPU and SPICE protocol, overcomes the problem of limited performance of traditional virtual graphics card, and is suitable for virtual machine application scenarios that require high-performance graphics processing. In addition, the present disclosure enables the QXL device to work normally, and the pictures displayed remotely by SPICE are rendered by virtual GPU, which makes full use of virtual GPU resources and improves the rendering ability of virtual machine graphics.

[0017] The above general description and the following description are exemplary and explanatory only and are not intended to limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] One or more embodiments are exemplarily described by corresponding drawings, which do not limit the embodiments. Elements with the same reference numerals in the drawings are shown as similar elements, and the drawings do not constitute a scale limitation, and wherein: Figure 1 It is a flowchart of an image rendering method based on the SPICE protocol and a virtual GPU provided by an embodiment of the present disclosure; Figure 2 It is a flowchart of another image rendering method based on SPICE protocol and virtual GPU provided by an embodiment of the present disclosure; Figure 3 It is a structural diagram of an image rendering system based on the SPICE protocol and a virtual GPU provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0019] In order to be able to understand the features and technical contents of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present disclosure. In the following technical description, for the convenience of explanation, a full understanding of the disclosed embodiments is provided through multiple details. However, one or more embodiments can still be implemented without these details. In other cases, to simplify the drawings, well-known structures and devices can be simplified for display.

[0020] The terms "first", "second", etc. in the embodiments of the present disclosure are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so as to describe the embodiments of the present disclosure described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions.

[0021] Unless otherwise stated, the term "plurality" means two or more.

[0022] In the embodiment of the present disclosure, the character " / " indicates that the preceding and following objects are in an "or" relationship. For example, A / B indicates: A or B.

[0023] The term "and / or" is a description of the association relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or, A and B.

[0024] The term "correspondence" may refer to an association relationship or a binding relationship. The correspondence between A and B means that there is an association relationship or a binding relationship between A and B.

[0025] The computer anomaly detection method and system provided by the embodiments of the present disclosure are described below with reference to the accompanying drawings.

[0026] Figure 1 It is a flowchart of an image rendering method based on the SPICE protocol and a virtual GPU provided in an embodiment of the present disclosure.

[0027] like Figure 1 As shown, the image rendering method based on the SPICE protocol and the virtual GPU may include: S01, using at least one physical computer as a node; enabling virtualization configuration on the physical computer node and installing at least one GPU device; installing a GPU driver and a GPU virtualization driver on the node, wherein the GPU driver is used to ensure the normal operation of the GPU, and the GPU virtualization driver is used to provide a virtual GPU for the virtual machine; creating at least one virtual machine on the node and assigning a virtual GPU to it; installing a virtual GPU driver in the virtual machine to ensure that the virtual GPU can normally perform rendering tasks; S02, installing a display driver model redirection driver in the virtual machine, the redirection driver being used to intercept and redirect rendering tasks of the virtual computer to the virtual GPU for processing; S03, installing the modified QXL driver in the virtual machine. The QXL driver will be recognized by the system as a PCI device driver, used to capture the image rendered by the virtual GPU, and transmit the graphics output of the virtual machine to the SPICE server through the SPICE protocol; S04, transmitting the image rendered by the virtual GPU to the SPICE client through the SPICE protocol to achieve remote display.

[0028] In the present disclosure, by combining virtual GPU and SPICE protocol, efficient graphics rendering and remote display are achieved, overcoming the problem of limited performance of traditional virtual graphics cards, and being suitable for virtual machine application scenarios that require high-performance graphics processing. In addition, the present disclosure enables the QXL device to work normally, and the pictures displayed remotely by SPICE are rendered by the virtual GPU, making full use of virtual GPU resources and improving the rendering capability of virtual machine graphics.

[0029] In some embodiments, the redirection driver is used to intercept and redirect the rendering task of the virtual computer to the virtual GPU for processing, including: The redirection driver intercepts all DirectX API calls sent from the operating system to the GPU by replacing the function pointer in the virtual function table of the DirectX object in the original function with a hook function. When the original function is called, the hook function is executed to intercept the rendering task.

[0030] In some embodiments, the method may include: When the hook function in the redirection driver intercepts a DirectX API call, it obtains the parameters of the API call, wherein the parameters of the API call describe a specific rendering task.

[0031] In some embodiments, the method may further include: The hook function in the redirection driver encapsulates the parameters of the API call into a task object that can be recognized by the virtual GPU, including setting the task type, rendering target, vertex data and texture.

[0032] In some embodiments, the method may further include: The hook function in the redirection driver calls the interface of the virtual GPU driver and adds the encapsulated task object to the task queue of the virtual GPU; After receiving the task, the virtual GPU executes the rendering task through its driver, and these tasks are executed on the hardware of the virtual GPU; After the virtual GPU completes the task, it generates the corresponding graphics output and passes it to the QXL driver through the shared memory mechanism.

[0033] In some embodiments, the modified QXL driver is identified as a PCI device driver by the system by defining a PCI device identifier, where the PCI device identifier includes a Vendor ID and a Device ID registered in the operating system.

[0034] In some embodiments, the modified QXL driver implements a probe function to handle device initialization, including enabling the device, requesting device resources, and mapping device memory space.

[0035] In some embodiments, the modified QXL driver defines a shared memory area to achieve data transmission between the virtual GPU driver and the QXL driver.

[0036] In some embodiments, the modified QXL driver defines an interface for obtaining graphics output to call the API of the virtual GPU driver to transfer data from the virtual GPU to the memory space managed by the QXL driver.

[0037] In some embodiments, the modified QXL driver compresses the graphic data using the H.264 compression algorithm and transmits the compressed graphic data to the SPICE server via the SPICE protocol.

[0038] Figure 2 is a flowchart of another image rendering method based on SPICE protocol and virtual GPU provided by the embodiment of the present disclosure, combined with Figure 2 ,right Figure 1 The image rendering method based on SPICE protocol and virtual GPU is further described in a specific example. Figure 1 The method may specifically include 1.1-1.4.

[0039] 1.1 Use at least one physical computer as a node; enable virtualization configuration on the physical computer node and install at least one GPU device; install a GPU driver and a GPU virtualization driver on the node, wherein the GPU driver is used to ensure the normal operation of the GPU and the GPU virtualization driver is used to provide a virtual GPU for the virtual machine; create at least one virtual machine on the node and allocate a virtual GPU to it; install a virtual GPU driver in the virtual machine to ensure that the virtual GPU can perform rendering tasks normally; 1.2 Install a Windows Display Driver Model (WDDM) redirection driver in the virtual machine, the redirection driver is used to intercept and redirect the rendering task of the virtual computer to the virtual GPU for processing; further, the WDDM redirection driver intercepts and redirects the rendering task of the virtual computer through hook functions. Specifically, the hook function intercepts all DirectX API calls sent from the operating system to the GPU by replacing the function pointer in the virtual function table (VTable) of the DirectX object in the original function. When the original function is called, the hook function is executed to intercept the rendering task. When the hook function in the WDDM redirection driver intercepts the DirectX API call, it first obtains the parameters of these API calls, specifically, these parameters describe the specific rendering task. The hook function in the WDDM redirection driver encapsulates these parameters into a task object that the virtual GPU can recognize, including setting the task type, rendering target, vertex data and texture, etc. The hook function in the WDDM redirection driver calls the interface of the virtual GPU driver and adds the encapsulated task object to the task queue of the virtual GPU. After receiving the task, the virtual GPU executes the rendering task through its driver, and these tasks are executed on the hardware of the virtual GPU. After the virtual GPU completes the task, it generates corresponding graphics output, which is passed to the QXL driver through the shared memory mechanism.

[0040] 1.3 Install the modified QXL driver in the virtual machine. The QXL driver will be identified by the system as a PCI device driver, which is used to capture the images rendered by the virtual GPU and transmit the graphics output of the virtual machine to the SPICE server through the SPICE protocol. Further, the modified QXL driver is identified by the system as a PCI device driver by defining PCI device identifiers. Specifically, the PCI device identifiers include Vendor ID and Device ID, which will be registered in the operating system. The modified QXL driver handles device initialization by implementing the probe function, specifically, including operations such as enabling the device, requesting device resources, and mapping device memory space. The modified QXL driver defines a shared memory area for efficient data transmission between the virtual GPU driver and the QXL driver. The modified QXL driver defines interfaces for obtaining graphics output. These interfaces can call the API of the virtual GPU driver to transfer data from the virtual GPU to the memory space managed by the QXL driver. The modified QXL driver uses the H.264 compression algorithm to compress graphics data, which significantly reduces the consumption of network bandwidth when transmitted to the SPICE server through the SPICE protocol, thereby improving the performance of remote display.

[0041] 1.4 Transmit the image rendered by the virtual GPU to the SPICE client through the SPICE protocol to achieve remote display.

[0042] Combination Figure 3 As shown, the embodiment of the present disclosure also provides an image rendering system 300 based on the SPICE protocol and virtual GPU, including a processor 304 and a memory 301. Optionally, the system may also include a communication interface 302 and a bus 303. Among them, the processor 304, the communication interface 302, and the memory 301 can communicate with each other through the bus 303. The communication interface 302 can be used for information transmission. The processor 304 can call the logic instructions in the memory 301 to execute the image rendering method based on the SPICE protocol and virtual GPU of the above embodiment.

[0043] In addition, the logic instructions in the memory 301 described above can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0044] The memory 301 is a computer-readable storage medium that can be used to store software programs and computer executable programs, such as program instructions / modules corresponding to the method in the embodiment of the present disclosure. The processor 304 executes the function application and data processing by running the program instructions / modules stored in the memory 301, that is, the image rendering method based on the SPICE protocol and the virtual GPU in the above embodiment is implemented.

[0045] The memory 301 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 301 may include a high-speed random access memory and may also include a non-volatile memory.

[0046] An embodiment of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured to execute an image rendering method based on a SPICE protocol and a virtual GPU.

[0047] The computer-readable storage medium mentioned above may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0048] The technical solution of the embodiment of the present disclosure can be embodied in the form of a software product, which is stored in a storage medium and includes one or more instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method of the embodiment of the present disclosure. The aforementioned storage medium may be a non-transient storage medium, including: a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, and other media that can store program codes, or a transient storage medium.

[0049] The above description and the accompanying drawings fully illustrate the embodiments of the present disclosure so that those skilled in the art can practice them. Other embodiments may include structural, logical, electrical, process and other changes. The embodiments represent possible changes only. Unless explicitly required, individual components and functions are optional, and the order of operations may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. As used in the description of the embodiments, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms as well. Similarly, the term "and / or" as used in this application refers to any and all possible combinations of one or more associated listings. In addition, when used in this application, the term "comprise" and its variants "comprises" and / or including (comprising) refer to the existence of stated features, wholes, steps, operations, elements, and / or components, but do not exclude the existence or addition of one or more other features, wholes, steps, operations, elements, components and / or these groups. In the absence of further restrictions, the elements defined by the sentence "comprising a ..." do not exclude the existence of other identical elements in the process, method or device comprising the elements. In this article, each embodiment may focus on the differences from other embodiments, and the same or similar parts between the embodiments may refer to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can refer to the description of the method part.

[0050] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. Technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present disclosure. Technicians can clearly understand that for the convenience and simplicity of description, the specific working process of the systems, devices and units described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0051] In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units can be only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to implement this embodiment. In addition, each functional unit in the embodiment of the present disclosure may be integrated in a processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit.

[0052] The flowchart and block diagram in the accompanying drawings show the possible architecture, functions and operations of the system, method and computer program product according to the embodiment of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and a part of the module, program segment or code contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. In the description corresponding to the flowchart and block diagram in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in a different order from the order disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified functions or actions, or may be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. An image rendering method based on SPICE protocol and virtual GPU, characterized in that: The method comprises: At least one physical computer is used as a node; virtualization configuration is enabled on the physical computer node, and at least one GPU device is installed; a GPU driver and a GPU virtualization driver are installed on the node, wherein the GPU driver is used to ensure the normal operation of the GPU, and the GPU virtualization driver is used to provide a virtual GPU for the virtual machine; at least one virtual machine is created on the node, and a virtual GPU is allocated to it; a virtual GPU driver is installed in the virtual machine to ensure that the virtual GPU can perform rendering tasks normally; Installing a display driver model redirection driver in the virtual machine, wherein the redirection driver is used to intercept and redirect the rendering task of the virtual computer to the virtual GPU for processing; Install the modified QXL driver in the virtual machine. The QXL driver will be recognized by the system as a PCI device driver to capture the virtual GPU rendering screen and transmit the virtual machine's graphics output to the SPICE server through the SPICE protocol. The images rendered by the virtual GPU are transmitted to the SPICE client via the SPICE protocol for remote display.

2. The method according to claim 1, characterized in that The redirection driver is used to intercept and redirect the rendering task of the virtual computer to the virtual GPU for processing, including: The redirection driver intercepts all DirectX API calls sent from the operating system to the GPU by replacing the function pointer in the virtual function table of the DirectX object in the original function with a hook function. When the original function is called, the hook function is executed to intercept the rendering task.

3. The method according to claim 2, characterized in that The method comprises: When the hook function in the redirection driver intercepts a DirectX API call, it obtains the parameters of the API call, wherein the parameters of the API call describe a specific rendering task.

4. The method according to claim 3, characterized in that The method further comprises: The hook function in the redirection driver encapsulates the parameters of the API call into a task object that can be recognized by the virtual GPU, including setting the task type, rendering target, vertex data and texture.

5. The method according to claim 4, characterized in that The method further comprises: The hook function in the redirection driver calls the interface of the virtual GPU driver and adds the encapsulated task object to the task queue of the virtual GPU; After receiving the task, the virtual GPU executes the rendering task through its driver, and these tasks are executed on the hardware of the virtual GPU; After the virtual GPU completes the task, it generates the corresponding graphics output and passes it to the QXL driver through the shared memory mechanism.

6. The method according to claim 1, characterized in that The modified QXL driver is identified as a PCI device driver by the system by defining a PCI device identifier, wherein the PCI device identifier includes a Vendor ID and a Device ID registered in the operating system.

7. The method according to claim 1, characterized in that The modified QXL driver handles device initialization by implementing a probe function, including enabling the device, requesting device resources, and mapping device memory space.

8. The method according to claim 1, characterized in that The modified QXL driver implements data transmission between the virtual GPU driver and the QXL driver by defining a shared memory area.

9. The method according to claim 1, characterized in that: The modified QXL driver defines an interface for obtaining graphics output to call the API of the virtual GPU driver to transfer data from the virtual GPU to the memory space managed by the QXL driver.

10. The method according to claim 1, characterized in that The modified QXL driver compresses the graphic data using the H.264 compression algorithm and transmits the compressed graphic data to the SPICE server via the SPICE protocol.

Images