Information processing method and virtual computing system

Abstract

The application discloses an information processing method, comprising: a virtual display intercepting access of an image processing unit driver running in a virtual machine to an image processing unit and display, and intercepting access information related to the display, wherein the virtual display is constructed in a virtual machine monitor; and the virtual display returning response information to the image processing unit driver according to the access information. The method can solve the problems of high maintenance and deployment cost and low compatibility of the virtual display scheme provided by the prior art.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to an information processing method, and also to a virtual computing system. Background Technology

[0002] In the cloud desktop field, to provide better graphics card performance, cloud service providers typically use a GPU (Graphics Processing Unit) pass-through virtualization solution, directly allocating the graphics card device to the customer's virtual machine. To capture the customer's desktop, a monitor device usually needs to be connected to the pass-through graphics card for the graphics card driver to function properly and display the image. To save costs in cloud desktop services, a virtual display solution is used.

[0003] There are two common solutions for implementing virtual displays:

[0004] 1. Hardware-based virtual display, commonly known as a screen lock device or HDMI / DP spoofing device. Its principle is to plug a device (such as...) into the HDMI or DP port of the graphics card. Figure 1 (③) In this case, the device communicates with the graphics card by mimicking the communication methods of a real monitor, notifying the graphics card that a monitor exists on the HDMI or DP port. The graphics card then outputs to this non-existent device. The main drawback of this approach is the high cost of maintenance and deployment, especially for cloud service providers. Deploying this hardware virtual monitor requires maintenance personnel to go to the data center, and long-term exposure to the data center environment, such as structural elements and vibrations, can lead to instability. Additionally, these virtual monitors only support certain fixed resolutions, requiring hardware replacement for upgrades.

[0005] 2. Virtual displays implemented within the operating system based on software drivers (such as...) Figure 1 (④) This type of software typically simulates a monitor within the operating system via a driver, and then uses system APIs or configuration files to cause the graphics card to output to this driver-simulated monitor. However, the problem with this method is that it needs to be adapted to different operating systems, such as Windows, different versions of Linux, etc.; at the same time, due to changes in the operating system API or configuration, compatibility issues may arise.

[0006] In summary, existing virtual display solutions suffer from high maintenance and deployment costs and low compatibility. Summary of the Invention

[0007] This application provides an information processing method to address the problems of high maintenance and deployment costs and low compatibility of existing virtual display solutions.

[0008] This application provides a data processing method, including:

[0009] The virtual display intercepts the image processing unit driver running in the virtual machine from accessing the image processing unit for display-related purposes, and captures the access information related to the display. The virtual display is built in the virtual machine monitor.

[0010] The virtual display returns response information to the image processing unit driver based on the access information.

[0011] As one implementation, the virtual display intercepts the image processing unit driver running in the virtual machine from accessing the image processing unit for display-related purposes, and intercepts display-related access information, including:

[0012] The virtual display intercepts the image processing unit driver in the virtual machine from accessing the image processing unit for display-related purposes and obtains access information related to the display.

[0013] The access path of the access information into the image processing unit is cut off;

[0014] The access information is processed.

[0015] As one implementation, the virtual display returns response information to the image processing unit driver based on the access information, including:

[0016] The virtual display returns its status information to the image processing unit based on the display detection information.

[0017] As one implementation, the virtual display returns response information to the image processing unit driver based on the access information, including:

[0018] The virtual display returns display information to the image processing unit based on the display query information.

[0019] In one implementation, the display information of the virtual display includes at least one of the following virtual display information:

[0020] The resolution of the virtual display;

[0021] The refresh rate of the virtual display;

[0022] The color format of the virtual display;

[0023] Bit depth of the virtual display.

[0024] This application also provides a virtual computing system, including: a virtual machine and a virtual machine monitor;

[0025] The image processing unit driver runs in the virtual machine;

[0026] A virtual display is built in the virtual machine monitor; wherein, the virtual display is used to intercept the image processing unit driver running in the virtual machine for display-related access to the image processing unit, and to intercept the access information related to the display, wherein, the virtual display is built in the virtual machine monitor; the virtual display returns response information to the image processing unit driver according to the access information.

[0027] In one implementation, the virtual display includes at least one of the following functional modules:

[0028] A virtual input / output device, the virtual input / output device being used to process read / write operation instructions from the image processing unit driver;

[0029] A virtual communication bus device, the virtual communication bus device being used to process communication commands driven by an image processing unit;

[0030] A virtual display information storage device, wherein the virtual display information storage device is used to store preset display information.

[0031] In one implementation, the virtual input / output device is a virtual memory-mapped input / output device.

[0032] In one implementation, the virtual communication bus device is a virtual I2C device or a virtual auxiliary channel device.

[0033] In one implementation, the virtual display information storage device is also used to communicate with the virtual machine monitor.

[0034] Compared with the prior art, this application has the following advantages:

[0035] This application provides an information processing method, including:

[0036] The virtual display intercepts the image processing unit driver running in the virtual machine from accessing the image processing unit for display-related purposes, and captures the access information related to the display. The virtual display is built in the virtual machine monitor. The virtual display returns response information to the image processing unit driver based on the access information.

[0037] The information processing method provided in this application constructs a virtual display in the virtual machine monitor layer. Compared with hardware-based virtual displays (such as screen lock devices and HDMI / DP spoofing devices), this application is a software solution based on the Hypervisor layer (virtual machine monitor layer), which does not require hardware support, thus avoiding the operation and maintenance costs of hardware-based solutions. Simultaneously, the software is easily upgradable, readily supporting more resolutions or multiple virtual displays simultaneously without requiring hardware upgrades or the installation of new hardware. Compared with virtual display solutions based on software drivers within a virtual machine, this application simulates hardware communication interfaces, eliminating the need for different virtual display software for different operating systems, thus reducing maintenance costs. Furthermore, since this application is implemented at the virtual machine monitor layer and is independent of the virtual machine operating system, it does not require reinstallation and deployment after operating system upgrades or replacements, avoiding software installation and deployment costs and solving the problem of low compatibility. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of an application scenario provided in the first embodiment of this application.

[0039] Figure 2 This is a flowchart of an information processing method provided in the first embodiment of this application.

[0040] Figure 3 This is a schematic diagram of a communication process provided in the first embodiment of this application. Detailed Implementation

[0041] Numerous specific details are set forth in the following description to provide a full understanding of the invention. However, the invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0042] To enable those skilled in the art to better understand the solution of this application, a specific application scenario embodiment of this application will be described in detail first.

[0043] like Figure 1 The diagram shown is an embodiment of an application scenario of an information processing method provided in this application. Figure 1 It can enable cloud desktops. Figure 1This includes a virtual machine, a hypervisor layer, and a GPU device. The virtual machine (VM) refers to a complete computer system simulated by software, possessing full hardware system functionality and running in a completely isolated environment. The hypervisor layer, also known as the virtual machine monitor (VMM), is the software, firmware, or hardware used to create and execute the virtual machine. The GPU device, or Graphics Processing Unit, is the image processing unit. The GPU driver within the virtual machine accesses the GPU device's I / O or MMIO, which can be divided into two categories: one related to the display, typically accessing the GPU's internal I2C bus to communicate with the display. Figure 1 The first category is represented by ②; the second category consists of other I / O or MMIO accesses, such as GPU task submission and GPU status query. Figure 1 The symbol ① indicates this. The technical solution proposed in this application adds a software virtual display (such as...) to the Hypervisor layer. Figure 1 (⑥) In this process, the software virtual display intercepts display-related I / O or MMIO operations in the GPU driver, preventing these I / O or MMIO operations from directly accessing the GPU hardware. Simultaneously, the software virtual display responds to these I / O or MMIO operations, returning virtual display information according to the communication logic of the real physical hardware, including but not limited to display resolution, refresh rate, color format, and bit depth. Thus, after the GPU driver obtains this information, it recognizes a real display existing on the GPU hardware, achieving the goal of virtualizing one or more displays.

[0044] The first embodiment of this application provides an information processing method. The following is in conjunction with... Figure 2 , Figure 3 Please provide an explanation.

[0045] like Figure 2 As shown, in step S201, the virtual display intercepts the image processing unit driver running in the virtual machine for display-related access to the image processing unit, and intercepts the display-related access information, wherein the virtual display is built in the virtual machine monitor.

[0046] The virtual machine monitor (VMM), or hypervisor layer, is the software, firmware, or hardware used to create and run virtual machines.

[0047] The virtual display may include at least one of the following functional modules:

[0048] A virtual input / output device, the virtual input / output device being used to process read / write operation instructions from the image processing unit driver;

[0049] A virtual communication bus device, the virtual communication bus device being used to process communication commands driven by an image processing unit;

[0050] A virtual display information storage device, wherein the virtual display information storage device is used to store preset display information.

[0051] Input / output typically refers to the input and output of data between memory (internal and external) or other peripheral devices. In this embodiment, virtual input / output devices can be constructed using software in a virtual machine monitor.

[0052] The virtual communication bus device may include a virtual I2C device or a virtual auxiliary channel device.

[0053] In existing technologies, GPU drivers within virtual machines accessing GPU devices' I / O or MMIO can be divided into two categories: one is related to the display, typically accessing the GPU's internal I2C bus to communicate with the physically existing display. Figure 1 This is represented by ②. Another category is other I / O or MMIO access, such as GPU task submission, GPU status query, etc. Figure 1 The I2C bus inside the GPU in the prior art is implemented by software in a virtual display, using the technical solution provided in this application.

[0054] In this embodiment, the Extended Display Identification Data (EEID) information of the display can be obtained through the I2C interface of the High Definition Multimedia Interface (HDMI) or the AUX channel of the DisplayPort interface. EEID is a VESA (Video Electronics Standards Association) standard data format that contains parameters related to the monitor and its performance, including vendor information, maximum image size, settings, manufacturer presets, frequency range limitations, and strings such as monitor name, resolution, refresh rate, color format, bit depth, and serial number.

[0055] I2C (Inter-Integrated Circuit) is a two-wire serial bus that has become the de facto standard for low-speed serial communication between chips and is widely used in consumer and control electronic devices. The DisplayPort (DP) interface is an image display interface that supports not only Full HD resolution (1920×1080), but also 4K resolution (3840×2160) and 8K resolution (7680×4320). The DP interface is not only high-speed but also reliable and stable. Its transmitted signals consist of data channel signals for transmitting images and auxiliary channel signals for transmitting image-related status and control information. Specifically, it includes the DisplayPort main data transmission channel (Main Link), auxiliary channel (AUX Channel), and link training. The AUX Channel, as an independent bidirectional auxiliary transmission channel in the DP interface, uses AC-coupled differential transmission and is a bidirectional half-duplex transmission channel with a single-direction rate of only about 1 Mbit / s, used to transmit setting and control commands. The AUX (Auxiliary) channel is used for reading Extended Display Identification Data (EDID) to ensure correct transmission of DP signals; reading information about the DP interfaces supported by the display, such as the number of main channels and the transmission rate of DP signals; setting various display configuration registers; and reading the display status registers.

[0056] In the first embodiment of this application, a virtual display information storage device is constructed in the virtual machine monitor layer. This virtual display information storage device stores preset display information, including extended display identifier data such as resolution, refresh rate, color format, and bit depth. Furthermore, the virtual display information storage device is also used to communicate with the virtual machine monitor to dynamically update the virtual display information.

[0057] The virtual display intercepts the image processing unit driver running in the virtual machine from accessing the image processing unit for display-related purposes, and intercepts display-related access information, including:

[0058] The virtual display intercepts the image processing unit driver in the virtual machine from accessing the image processing unit and display-related information, and obtains access information related to the display.

[0059] The access path of the access information into the image processing unit is cut off;

[0060] The access information is processed.

[0061] like Figure 3As shown, it is a schematic diagram of the communication process between "GPU driver - Hypervisor layer virtual display - GPU hardware".

[0062] The access information related to the display includes the status information of the virtual display, such as whether the virtual display exists. Figure 3 In this context, a virtual display can be used to simulate the presence of a physical monitor. Additionally, display-related access information includes virtual display-related data such as resolution, refresh rate, color format, bit depth, and other extended display identifier data. Figure 3 In this context, a virtual display can simulate monitor information. By simulating its existence and monitor information, the functionality of a cloud desktop is achieved.

[0063] like Figure 3 As shown, when the GPU driver attempts to detect the display or attempt to obtain display information, the virtual display set in the Hypervisor layer will intercept the image processing unit driver's access information to the image processing unit device, prevent it from directly accessing the hardware, process the relevant access, and return the status information or related information of the virtual display.

[0064] The virtual display processes the access information, including:

[0065] The virtual display processes the access information through a pre-registered callback function.

[0066] A callback function is a function that is passed as an argument. In C, callback functions can only be implemented using function pointers; in languages ​​like C++ and Python, anonymous functions can also be used. Using callback functions can improve programming efficiency. In this embodiment, the virtual display processes the access information through a pre-registered callback function to obtain information about whether the virtual display exists and the display itself.

[0067] like Figure 2 As shown, in step S202, the virtual display returns response information to the image processing unit driver based on the access information.

[0068] The virtual display returns response information to the image processing unit driver based on the access information, including:

[0069] The virtual display returns its status information to the image processing unit based on the display detection information.

[0070] The virtual display returns response information to the image processing unit driver based on the access information, including:

[0071] The virtual display returns display information to the image processing unit based on the display query information.

[0072] The display information of the virtual display includes at least one of the following virtual display information:

[0073] The resolution of the virtual display;

[0074] The refresh rate of the virtual display;

[0075] The color format of the virtual display;

[0076] Bit depth of the virtual display.

[0077] The virtual display returns its status information to the image processing unit based on the display detection information. For example... Figure 3 As shown, the image processing unit driver detects the presence of the display through the Hypervisor layer. The virtual display intercepts the display detection information access and directly returns a message indicating that the display exists to the image processing unit driver. This detection information access is not transmitted to the image processing unit.

[0078] The virtual display, based on the display query information, drives the image processing unit to return display information of the virtual display. For example... Figure 3 As shown, the image processing unit driver queries the display information through the Hypervisor layer. The virtual display intercepts the display's query information access and directly returns the display information to the image processing unit driver. The query information access is not transmitted to the image processing unit.

[0079] For pass-through memory-mapped I / O (MMIO), i.e. Figure 3For other types of access (such as GPU task submission and GPU status query), the Hypervisor layer modifies the extended page-table (EPT) mapping table to directly map the guest physical address of these MMIOs to the host physical address of the physical MMIO. This way, when the virtual machine accesses these MMIOs, it directly accesses the physical MMIO. For MMIOs that need to be intercepted, the Hypervisor layer typically does not perform EPT mapping (i.e., page fault handling). When the virtual machine accesses these MMIOs, it will be trapped in the Hypervisor layer. At this point, the virtual monitor can process these MMIO accesses by registering callback functions, and the processed values ​​are then returned to the virtual machine.

[0080] This concludes the introduction of the first embodiment of this application. The information processing method provided in the first embodiment of this application constructs a virtual display in the virtual machine monitor layer. Compared with hardware-based virtual displays (such as screen lock devices, HDMI / DP spoofing devices, etc.), this application is a software solution based on the Hypervisor layer (virtual machine monitor layer), which does not require hardware support, thus avoiding the operation and maintenance costs of hardware-based solutions. Simultaneously, the software is easily upgradable, readily supporting more resolutions or multiple virtual displays simultaneously without requiring hardware upgrades or the installation of new hardware. Compared with virtual display solutions implemented based on software drivers within a virtual machine, this application simulates hardware communication interfaces, eliminating the need to provide different virtual display software for different operating systems, thus reducing maintenance costs. Furthermore, since this application is implemented at the virtual machine monitor layer and is independent of the virtual machine operating system, it does not require reinstallation and deployment after operating system upgrades or replacements, avoiding software installation and deployment costs and solving the problem of low compatibility.

[0081] A second embodiment of this application provides a virtual computing system, including: a virtual machine and a virtual machine monitor;

[0082] The image processing unit driver runs in the virtual machine;

[0083] A virtual display is built in the virtual machine monitor; wherein, the virtual display is used to intercept the image processing unit driver running in the virtual machine for display-related access to the image processing unit, and to intercept the access information related to the display, wherein, the virtual display is built in the virtual machine monitor; the virtual display returns response information to the image processing unit driver according to the access information.

[0084] In one implementation, the virtual display includes at least one of the following functional modules:

[0085] A virtual input / output device, the virtual input / output device being used to process read / write operation instructions from the image processing unit driver;

[0086] A virtual communication bus device, the virtual communication bus device being used to process communication commands driven by an image processing unit;

[0087] A virtual display information storage device, wherein the virtual display information storage device is used to store preset display information.

[0088] In one implementation, the virtual input / output device is a virtual memory-mapped input / output device.

[0089] In one implementation, the virtual communication bus device is a virtual I2C device or a virtual auxiliary channel device.

[0090] In one implementation, the virtual display information storage device is also used to communicate with the virtual machine monitor.

[0091] The virtual computing system provided in the second embodiment of this application constructs a virtual display in the virtual machine monitor layer. Compared with hardware-based virtual displays (such as screen lock devices, HDMI / DP spoofing devices, etc.), this application is based on a software solution at the Hypervisor layer (virtual machine monitor layer), which does not require hardware support, thus avoiding the operation and maintenance costs of hardware-based solutions. Simultaneously, the software is easily upgradable, readily supporting more resolutions or multiple virtual displays simultaneously without requiring hardware upgrades or the installation of new hardware. Compared with virtual display solutions implemented within a virtual machine based on software drivers, this application simulates hardware communication interfaces, eliminating the need to provide different virtual display software for different operating systems, thus reducing maintenance costs. Furthermore, since this application is implemented at the virtual machine monitor layer and is independent of the virtual machine operating system, it does not require reinstallation and deployment after operating system upgrades or replacements, avoiding software installation and deployment costs and solving the problem of low compatibility.

[0092] It should be noted that the virtual computing system provided in the second embodiment of this application is a scenario embodiment of the first embodiment of this application, and its detailed description can be found in the relevant description of the first embodiment of this application, which will not be repeated here.

[0093] Although this application discloses preferred embodiments as described above, it is not intended to limit this application. Any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of this application. Therefore, the scope of protection of this application should be determined by the scope defined in the claims of this application.

[0094] In a typical configuration, a computing device includes one or more processors (CPUs), memory-mapped input / output interfaces, network interfaces, and memory.

[0095] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0096] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include non-transitory computer-readable media, such as modulated data signals and carrier waves.

[0097] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

Claims

1. An information processing method, characterized in that, include: The virtual display intercepts the image processing unit driver running in the virtual machine from accessing the image processing unit for display-related information, and captures the access information related to the display. The virtual display is built in the virtual machine monitor and is used to simulate display information. The virtual display returns response information to the image processing unit driver based on the access information, wherein the response information includes the status information and display information of the virtual display.

2. The method according to claim 1, characterized in that, The virtual display intercepts the image processing unit driver running in the virtual machine from accessing the image processing unit for display-related purposes, and intercepts display-related access information, including: The virtual display intercepts the image processing unit driver in the virtual machine from accessing the image processing unit for display-related purposes and obtains access information related to the display. The access path of the access information into the image processing unit is cut off; The access information is processed.

3. The method according to claim 1, characterized in that, The virtual display returns response information to the image processing unit driver based on the access information, including: The virtual display returns its status information to the image processing unit based on the display detection information.

4. The method according to claim 1, characterized in that, The virtual display returns response information to the image processing unit driver based on the access information, including: The virtual display returns display information to the image processing unit based on the display query information.

5. The method according to claim 4, characterized in that, The display information of the virtual display includes at least one of the following virtual display information: The resolution of the virtual display; The refresh rate of the virtual display; The color format of the virtual display; Bit depth of the virtual display.

6. A virtual computing system, characterized in that, include: Virtual machines and virtual machine monitors; The image processing unit driver runs in the virtual machine; A virtual display is built in the virtual machine monitor; wherein, the virtual display is used to intercept the image processing unit driver running in the virtual machine for display-related access to the image processing unit, and to intercept the access information related to the display, wherein the virtual display is built in the virtual machine monitor; the virtual display returns response information to the image processing unit driver according to the access information, the virtual display is used to simulate display information, and the response information includes the status information and display information of the virtual display.

7. The virtual computing system according to claim 6, characterized in that, The virtual display includes at least one of the following functional modules: A virtual input / output device, the virtual input / output device being used to process read / write operation instructions from the image processing unit driver; A virtual communication bus device, the virtual communication bus device being used to process communication commands driven by an image processing unit; A virtual display information storage device, wherein the virtual display information storage device is used to store preset display information.

8. The virtual computing system according to claim 7, characterized in that, The virtual input / output device is a virtual memory-mapped input / output device.

9. The virtual computing system according to claim 7, characterized in that, The virtual communication bus device is a virtual I2C device or a virtual auxiliary channel device.

10. The virtual computing system according to claim 7, characterized in that, The virtual display information storage device is also used to communicate with the virtual machine monitor.

Images