Virtualized screen projection method, virtualized screen projection hardware device and storage medium

Abstract

The application provides a virtual projection method, a hardware device for virtual projection and a storage medium. A privileged virtual machine can first create a display virtual window corresponding to a guest virtual machine, display the display virtual window in a preset screen corresponding to the guest virtual machine, and create a transparent virtual window located below the display virtual window. The corresponding video memory physical address is obtained, and then the same display image as the display virtual window is rendered for the transparent virtual window in response to the rendering instruction issued by the guest virtual machine, so that the display image is stored in the video memory area corresponding to the video memory physical address. In response to the display instruction issued by the guest virtual machine, the display image is obtained from the video memory area, and the display image is sent to the projection device for projection display, realizing projection of the content displayed by the guest virtual machine on the screen. It can flexibly adapt to different resolution, hardware interface projection devices, and also can ensure that each frame of image of the projection is not distorted.

Description

Technical Field

[0001] This application relates to the field of projection technology, specifically to a virtualized projection method, a virtualized projection hardware device, and a storage medium. Background Technology

[0002] Currently, hardware virtualization technology allows multiple operating systems or virtual machines to run simultaneously on the same hardware platform, using a hypervisor to create a secure and independent software runtime environment on the underlying hardware. Through a hypervisor, the in-vehicle infotainment system can access multiple operating systems without integrating two or more hardware devices.

[0003] However, to project content onto a screen, existing methods typically require introducing a display processing unit, which then needs to add an interface to connect to a projection device. This approach is costly and difficult to adapt flexibly to different interfaces or types of projection devices. Summary of the Invention

[0004] In view of the above-mentioned defects or deficiencies in the prior art, this application aims to provide a virtualized projection method, a virtualized projection hardware device and a storage medium to solve the problems of high projection cost and difficulty in flexibly adapting to different interfaces or different types of projection devices in the prior art.

[0005] This application provides a virtualized screen projection method, executed by a privileged virtual machine in a hardware device, wherein the hardware device further includes at least one guest virtual machine, and the method includes:

[0006] Create a virtual display window corresponding to the client virtual machine, and display the virtual display window in the preset screen corresponding to the client virtual machine;

[0007] Create a transparent virtual window located below the display virtual window, and obtain the physical address of the video memory corresponding to the transparent virtual window;

[0008] In response to the rendering command issued by the client virtual machine, a display image identical to that of the display virtual window is rendered for the transparent virtual window, so that the display image is stored in the video memory area corresponding to the video memory physical address;

[0009] In response to the display command issued by the client virtual machine, the display image is obtained from the video memory area corresponding to the physical address of the video memory, and the display image is sent to the projection device for projection display.

[0010] Optionally, the step of creating a display virtual window corresponding to the guest virtual machine includes:

[0011] Obtain the display port number of the preset screen corresponding to the client virtual machine, and determine the resolution of the preset screen based on the display port number;

[0012] A virtual display window corresponding to the client virtual machine is created based on the preset screen resolution.

[0013] Optionally, creating a transparent virtual window located below the displayed virtual window includes:

[0014] The resolution of the projection device is determined based on the configuration file, and a virtual window is created according to the resolution of the projection device;

[0015] Set the virtual window to transparent to obtain a transparent virtual window, and place the transparent virtual window below the displayed virtual window.

[0016] Optionally, in response to a rendering command issued by the client virtual machine, rendering a display image for the transparent virtual window that is identical to that of the display virtual window includes:

[0017] In response to the rendering command issued by the guest virtual machine, the virtualization graphics processing dynamic library is invoked to parse the rendering command;

[0018] After parsing is completed, graphics processing instructions are submitted to the graphics processor through the interface in the rendering management library, so that the graphics processor renders and synthesizes the image according to the display image of the display virtual window, so that the transparent virtual window displays the same display image as the display virtual window.

[0019] Optionally, in response to a display command issued by the guest virtual machine, the display image is obtained from the video memory region corresponding to the physical address of the video memory, including:

[0020] In response to the display command issued by the client virtual machine, the interface in the graphics service subsystem is invoked to enable the graphics service subsystem to initiate direct memory access and read the display image from the video memory area corresponding to the transparent virtual window according to the video memory physical address.

[0021] Optionally, the rendering instruction and the display instruction are generated by the graphics service that calls the virtual graphics device interface of the client virtual machine.

[0022] Optionally, before creating the display virtual window corresponding to the guest virtual machine, the method further includes:

[0023] Load the graphics processor driver and start the graphics service subsystem;

[0024] Start the guest virtual machine so that it loads the driver for the virtual graphics device interface and starts the graphics service.

[0025] This application embodiment also provides a hardware device for virtualized screen projection, the hardware device including a privileged virtual machine and at least one guest virtual machine, wherein;

[0026] The client virtual machine is used to issue rendering instructions and display instructions to the privileged virtual machine;

[0027] The privileged virtual machine is used to create a display virtual window corresponding to the client virtual machine, display the display virtual window within a preset screen corresponding to the client virtual machine, create a transparent virtual window below the display virtual window, obtain the physical address of the video memory corresponding to the transparent virtual window, and, in response to a rendering command issued by the client virtual machine, render a display image identical to the display virtual window for the transparent virtual window, so that the display image is stored in the video memory area corresponding to the physical address of the video memory. In response to a display sending command issued by the client virtual machine, the display image is obtained from the video memory area corresponding to the physical address of the video memory and sent to a projection device for projection display.

[0028] Optionally, the privileged virtual machine includes a device model and a graphics service subsystem, wherein;

[0029] The device model is used to call the graphics service subsystem;

[0030] The graphics service subsystem is used to create a display virtual window corresponding to the client virtual machine, create a transparent virtual window located below the display virtual window, and read the display image from the video memory area corresponding to the transparent virtual window according to the video memory physical address.

[0031] This application embodiment also provides an electronic device, the electronic device comprising:

[0032] Processor and memory;

[0033] The processor executes the steps of the virtualization projection method provided in any embodiment of this application by calling the program or instructions stored in the memory.

[0034] This application also provides a computer-readable storage medium that stores a program or instructions that cause a computer to execute the steps of the virtualization screen projection method provided in any embodiment of this application.

[0035] In summary, this application proposes a virtualized screen projection method. This method is executed by a privileged virtual machine in a hardware device. The privileged virtual machine can first create a display virtual window corresponding to the guest virtual machine, display the display virtual window within a preset screen corresponding to the guest virtual machine, and create a transparent virtual window located below the display virtual window. It then obtains the physical address of the video memory corresponding to the transparent virtual window, and responds to rendering instructions issued by the guest virtual machine to render a display image identical to the display virtual window for the transparent virtual window. This ensures that the display image is stored in the video memory area corresponding to the physical address of the video memory. Finally, in response to display sending instructions issued by the guest virtual machine, the image is sent from the display memory area corresponding to the physical address of the video memory. The method acquires the display image from the region and sends it to a projection device for projection. By setting a transparent virtual window and the corresponding physical address of video memory, it achieves the projection of the content displayed on the screen by the client virtual machine. Since the transparent virtual window is located below the display virtual window and is transparent, it will not affect the display of the display virtual window on the preset screen, thus avoiding the projection from affecting the content displayed on the screen by the client virtual machine. Furthermore, the virtual projection implemented by this method can flexibly adapt to projection devices with different resolutions and hardware interfaces without adding new interfaces to the projection device. It can also ensure that each frame of the projected image is not distorted, improving the user experience. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0037] Figure 1 This is a flowchart of a virtualized screen projection method provided in an embodiment of this application;

[0038] Figure 2 This is a flowchart illustrating screen projection based on virtualization technology, provided in an embodiment of this application.

[0039] Figure 3 This is a system block diagram of a hardware device provided in an embodiment of this application;

[0040] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0041] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0042] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0043] As mentioned in the background section, this application proposes a virtualized screen projection method to address the problems in the existing technology. This method can be applied to projecting content displayed on the screen of a client virtual machine in an in-vehicle infotainment system, or it can be applied to projecting content displayed on the screen of a client virtual machine in a computer, etc.

[0044] The virtualized screen projection method provided in this application can be executed by a privileged virtual machine in a hardware device, which also includes at least one guest virtual machine. The hardware device can be an electronic device capable of hardware virtualization, such as an in-vehicle infotainment system or a computer. A guest virtual machine is a virtual machine running in the virtualization environment of a hardware device. A hardware device can create multiple isolated virtual computer environments through virtualization software. These virtual computer environments, referred to as guest virtual machines, share the host machine's hardware resources, such as memory, storage, and network. Each guest virtual machine behaves like an independent physical computer at the operating system and application levels. A privileged virtual machine is a virtual machine with special permissions in the virtualization environment of a hardware device, capable of performing operations crucial to system resource management and security control.

[0045] Figure 1 This is a flowchart of a virtualized screen projection method provided in an embodiment of this application. See also... Figure 1 The virtualized screen mirroring method specifically includes:

[0046] S110. Create a display virtual window corresponding to the guest virtual machine and display the display virtual window in the preset screen corresponding to the guest virtual machine.

[0047] The display virtual window is a virtual window used to display the running content of the guest virtual machine in full screen. Specifically, a display virtual window corresponding to the guest virtual machine can be created according to the specified display screen of the guest virtual machine, so that the display virtual window can adapt to the resolution of the specified display screen of the guest virtual machine.

[0048] In one example, creating a display virtual window corresponding to the guest virtual machine includes:

[0049] Obtain the display port number of the preset screen corresponding to the guest virtual machine, and determine the resolution of the preset screen based on the display port number; create a display virtual window corresponding to the guest virtual machine based on the resolution of the preset screen.

[0050] The preset screen corresponding to the guest virtual machine can be a pre-specified screen that displays the content running on the guest virtual machine. The display port number of the preset screen can be the DP (DisplayPort) port number of the preset screen.

[0051] For example, the correspondence between different customer virtual machines and each screen in the vehicle system can be written into a table in advance, so as to query the preset screen corresponding to each customer virtual machine and the display port number of the preset screen in the table.

[0052] Furthermore, the preset screen resolution can be obtained based on the display port number. Resolution refers to the number of pixels contained in the screen horizontally and vertically, usually expressed as "horizontal pixels × vertical pixels". Different screens in the vehicle's infotainment system can have their own resolutions. For example, the correspondence between different screens and their resolutions can be pre-written in a table, allowing for easy lookup of the resolution for each screen.

[0053] After obtaining the preset screen resolution, a display virtual window that meets that resolution can be created to display the running content of the guest virtual machine on the preset screen.

[0054] The above example demonstrates how the resolution can be determined based on the display port number of the preset screen corresponding to the guest virtual machine. This allows for the creation of a display virtual window that meets the specified resolution, ensuring that the display virtual window can adapt to the preset screen and thus guaranteeing the display effect within the preset screen.

[0055] Specifically, after creating the display virtual window corresponding to the guest virtual machine, the display virtual window can be displayed within the preset screen corresponding to the guest virtual machine.

[0056] S120. Create a transparent virtual window located below the display virtual window, and obtain the physical address of the video memory corresponding to the transparent virtual window.

[0057] Specifically, after creating the display virtual window within the preset screen, a transparent virtual window corresponding to the projection device can be created below the display virtual window. The projection device can be an external projector connected to the hardware device.

[0058] In one example, creating a transparent virtual window located below the displayed virtual window includes:

[0059] The resolution of the projection device is determined based on the configuration file, and a virtual window is created according to the resolution of the projection device; the virtual window is set to transparent display to obtain a transparent virtual window, and the transparent virtual window is placed below the displayed virtual window.

[0060] The configuration file can be a file used to transfer parameter information between devices. Specifically, the resolution of the connected projection device can be read from the configuration file.

[0061] Furthermore, a virtual window can be created based on the resolution of the projection device, so that the virtual window can adapt to the resolution of the projection device. After creating the virtual window, its property can be set to transparent display to obtain a transparent virtual window, and then this virtual window can be set as the bottom layer display and placed below the displayed virtual window.

[0062] The above example demonstrates how to create a transparent virtual window below the display virtual window based on the resolution of the projection device, ensuring that the transparent virtual window meets the resolution of the projection device and reflects the display effect of the projection device.

[0063] It should be noted that in this embodiment, the transparent virtual window is set below the display virtual window. The purpose is that the display on a screen can be divided into multiple layers, each of which can have different transparency. By setting a transparent virtual window below the virtual window, the transparent virtual window corresponding to the projection device can be prevented from affecting the display of the display virtual window, thereby avoiding the impact of projection on the content displayed on the screen and ensuring the display effect of the screen during the projection process.

[0064] After creating a transparent virtual window located below the displayed virtual window, the physical address of the corresponding video memory can be obtained. This involves pre-allocating video memory (display memory) to the graphics processing unit (GPU) in the hardware device. The physical address of this video memory can be the address of a memory region allocated for the transparent virtual window within the GPU's corresponding video memory.

[0065] S130. In response to the rendering command issued by the guest virtual machine, render the same display image as the display virtual window for the transparent virtual window, so that the display image is stored in the video memory area corresponding to the physical address of the video memory.

[0066] In this embodiment, if the client virtual machine detects that the user has triggered the projection control on the preset screen, or if the client virtual machine detects the user's projection voice command, it can send a rendering command to the privileged virtual machine to start the projection.

[0067] Specifically, in response to rendering commands issued by the client virtual machine, the images that need to be displayed in the client virtual machine application can be rendered and composited so that the transparent virtual window and the corresponding display virtual window of the client virtual machine display the same content. It should be noted that while the transparent virtual window and the display virtual window display the same content, their resolutions can be the same or different. The resolution of the transparent virtual window corresponds to the resolution of the projection device, while the resolution of the display virtual window corresponds to the resolution of the preset screen.

[0068] In one specific implementation, in response to a rendering command issued by the guest virtual machine, rendering a display image identical to that of the display virtual window for the transparent virtual window includes the following steps:

[0069] Step 11: In response to the rendering command issued by the guest virtual machine, call the virtualization graphics processing dynamic library to parse the rendering command;

[0070] Step 12: After parsing is complete, submit graphics processing instructions to the graphics processor through the interface in the rendering management library, so that the graphics processor renders and composites according to the display image of the virtual window, so that the transparent virtual window displays the same display image as the display virtual window.

[0071] In step 11, if a rendering instruction is received from the client virtual machine, the virtualization graphics processing dynamic library (Virglrenderer) can be called first to parse the rendering instruction.

[0072] Specifically, the Virtualized Graphics Processing Dynamic Library (VGMDL) designs a virtual OpenGL rendering device. The guest virtual machine can send OpenGL rendering instructions to this virtual device. Furthermore, the VGMDL parses the rendering instructions so that they can be passed to the graphics processor of the hardware device for accelerated processing.

[0073] Furthermore, in step 12, after parsing is complete, the graphics processor driver can be called through the interface in the rendering management library (libdrm) to submit graphics processing instructions to the graphics processor, so that the graphics processor can render and composite according to the display image of the virtual window, enabling the transparent virtual window to display the same display image as the virtual window.

[0074] Through steps 11-12 above, OpenGL (Open Graphics Library) calls in the guest virtual machine can be converted into instructions that the GPU in the hardware device can understand, thereby enabling the GPU in the hardware device to perform rendering and compositing, ensuring rendering efficiency.

[0075] It should be noted that the display image rendered for the transparent virtual window is stored in the video memory area corresponding to the transparent virtual window, that is, in the video memory area corresponding to the physical address of the transparent virtual window. The video memory area corresponding to the physical address can be the storage area pointed to by the physical address.

[0076] S140: In response to the display command issued by the client virtual machine, retrieve the display image from the video memory area corresponding to the physical address of the video memory, and send the display image to the projection device for projection display.

[0077] Specifically, after rendering and compositing the display image of the virtual window, the client virtual machine can continue to issue a display command. In response to this display command, the display image within the transparent virtual window can be sent to a projection device for projection display.

[0078] In one specific implementation, in response to a display command issued by the guest virtual machine, the display image is retrieved from the video memory region corresponding to the physical address of the video memory, including:

[0079] In response to the display command issued by the guest virtual machine, the interface in the graphics service subsystem is called to enable the graphics service subsystem to start direct memory access and read the display image from the video memory area corresponding to the transparent virtual window according to the physical address of the video memory.

[0080] Specifically, if a display command is received from a guest virtual machine, the interface in the graphics service subsystem can be called. The graphics service subsystem will then trigger Direct Memory Access (DMA) to transfer the data, access the physical address of the video memory, and read the display image from the corresponding video memory area.

[0081] It should be noted that during the projection of the content displayed on the preset screen by the guest virtual machine, the above-mentioned rendering and compositing steps, as well as the steps of reading and transmitting the display image from the video memory area, can be executed cyclically. For example, at the first moment of starting the projection, a frame of display image can be obtained from the first address in the video memory physical address. At the next moment, the next frame of image can be obtained from the next address in the video memory physical address, and so on, until the projection ends.

[0082] Therefore, the display frame rate of the projection device is the same as the frame rate of the client virtual machine application. For example, if the frame rate of the client virtual machine application is 30 frames / s, the graphics service subsystem will transmit 30 frames of images per second to the projection device for display via DMA.

[0083] In the above implementation, by calling the graphics service subsystem for DMA transfer, the video memory area allocated for the transparent virtual window in the GPU video memory can be directly accessed, thereby obtaining and transmitting the display image without the intervention of the CPU in the hardware device, which can improve the transmission efficiency of image data and greatly improve the utilization of the CPU.

[0084] Specifically, after reading the display image from the video memory area corresponding to the transparent virtual window, the display image can be sent to a projection device, which can then project the received display image, such as onto the windshield of the vehicle, onto the top of the vehicle, or onto a screen in the external environment of the vehicle.

[0085] In this embodiment, the rendering and display commands described above can be commands that conform to the Virtual Device Interface Specification (virtio-gpu). In one example, the rendering and display commands are generated by the guest virtual machine calling the graphics service of the virtual graphics device interface.

[0086] Specifically, the guest virtual machine can call the graphics services of the Virtual Graphics Device Interface (virtio-gpu) to generate rendering and display commands. In this embodiment, both the guest virtual machine and the privileged virtual machine can configure the Virtual Graphics Device Interface. By calling the Virtual Graphics Device Interface to generate commands, the virtual machine can better handle graphics-related tasks.

[0087] The aforementioned graphics service subsystem and virtual graphics device interface can all be loaded after the privileged virtual machine starts. Optionally, before creating the display virtual window corresponding to the guest virtual machine, the following also applies:

[0088] Load the graphics processor driver and start the graphics service subsystem; start the guest virtual machine so that it loads the virtual graphics device interface driver and starts the graphics service.

[0089] Specifically, the privileged virtual machine can first load the graphics processor driver in the hardware device and start the graphics service subsystem, so that it can subsequently create virtual windows, obtain the physical address of video memory, and perform rendering and compositing through the graphics processor.

[0090] Furthermore, the privileged virtual machine can start the guest virtual machine. After the guest virtual machine starts, it can load the driver for the virtual graphics device interface and then start the graphics service so that rendering instructions and display instructions can be generated through the graphics service.

[0091] For example, Figure 2 This is a flowchart illustrating screen projection based on virtualization technology, provided in an embodiment of this application. For example... Figure 2 As shown, firstly, the privileged virtual machine can load the graphics processor driver and start the graphics service subsystem. Then, the device model (DM) in the privileged virtual machine starts the guest virtual machine. The guest virtual machine loads the driver for the virtual graphics device interface (virtio-gpu), and then the guest virtual machine starts the graphics service.

[0092] After completing the preparations for the above services, see Figure 2 The entire screen mirroring process can include the following steps:

[0093] S1: The device model calls the graphics service subsystem to create a display virtual window for the guest virtual machine;

[0094] S2: The device model calls the graphics service subsystem to create a transparent virtual window for the projection device;

[0095] S3: The graphics service subsystem obtains the physical address of the video memory corresponding to the transparent virtual window;

[0096] S4: The device model waits for instructions from the guest virtual machine for processing. In this process, the virtual graphics device interface (virtio-gpu) in the device model receives rendering instructions, and the virtual graphics device interface (virtio-gpu) in the device model receives display instructions.

[0097] S5: The client virtual machine issues rendering instructions. After receiving the virtio-gpu rendering instructions, DM calls Virglrenderer to parse them, and then calls the GPU driver through the relevant interfaces in libdrm to submit relevant instructions to the GPU for rendering and compositing.

[0098] S6: The client virtual machine issues rendering instructions. After receiving the virtio-gpu display instruction, the DM calls the interface of the graphics service subsystem to display the graphics.

[0099] S7: The graphics service subsystem copies the image to the projection device via DMA, and the projection device projects the image.

[0100] In steps S1 through S7 above, a display virtual window needs to be created for the guest virtual machine first. This window is then displayed full-screen on the physical screen. Next, a transparent virtual window is created based on the projection device's resolution. This newly created transparent virtual window is set to be the bottom layer and transparent to avoid interfering with the guest virtual machine's display virtual window. The transparent virtual window can be rendered using virtio-gpu to produce the same content as the display virtual window. The graphics service subsystem obtains the physical address of the transparent virtual window's video memory. This address is where each rendered frame of image is stored. The graphics service subsystem then uses DMA to transfer each rendered frame of image to the projection device for screen projection.

[0101] The virtualized screen projection method provided in this application embodiment is executed by a privileged virtual machine in a hardware device. The privileged virtual machine can first create a display virtual window corresponding to the guest virtual machine, display the display virtual window within a preset screen corresponding to the guest virtual machine, and create a transparent virtual window located below the display virtual window. It then obtains the physical address of the video memory corresponding to the transparent virtual window, and in response to rendering instructions issued by the guest virtual machine, renders the same display image as the display virtual window for the transparent virtual window, so that the display image is stored in the video memory area corresponding to the physical address of the video memory. Finally, in response to a display sending instruction issued by the guest virtual machine, the image is sent from the video memory area corresponding to the physical address of the video memory. The method acquires the display image and sends it to a projection device for projection. By setting a transparent virtual window and the corresponding physical address of video memory, it achieves the projection of the content displayed on the screen by the client virtual machine. Since the transparent virtual window is located below the display virtual window and is transparent, it will not affect the display of the display virtual window on the preset screen, thus avoiding the projection from affecting the content displayed on the screen by the client virtual machine. Furthermore, the virtual projection implemented by this method can flexibly adapt to projection devices with different resolutions and hardware interfaces without adding new interfaces to the projection device. It can also ensure that each frame of the projected image is not distorted, improving the user experience.

[0102] For the same purpose, this application also provides a hardware device for virtualized screen projection, the hardware device including a privileged virtual machine and at least one guest virtual machine, wherein;

[0103] The client virtual machine is used to issue rendering instructions and display instructions to the privileged virtual machine;

[0104] The privileged virtual machine is used to create a display virtual window corresponding to the client virtual machine, display the display virtual window within a preset screen corresponding to the client virtual machine, create a transparent virtual window below the display virtual window, obtain the physical address of the video memory corresponding to the transparent virtual window, and, in response to a rendering command issued by the client virtual machine, render a display image identical to the display virtual window for the transparent virtual window, so that the display image is stored in the video memory area corresponding to the physical address of the video memory. In response to a display sending command issued by the client virtual machine, the display image is obtained from the video memory area corresponding to the physical address of the video memory and sent to a projection device for projection display.

[0105] Optionally, the privileged virtual machine includes a device model and a graphics service subsystem, wherein the device model is used to call the graphics service subsystem; the graphics service subsystem is used to create a display virtual window corresponding to the guest virtual machine, and to create a transparent virtual window located below the display virtual window, and to read the display image from the video memory area corresponding to the transparent virtual window according to the video memory physical address.

[0106] For example, the device model is used to obtain the display port number of the preset screen corresponding to the client virtual machine, determine the resolution of the preset screen based on the display port number, and call the graphics service subsystem so that the graphics service subsystem creates a display virtual window corresponding to the client virtual machine based on the resolution of the preset screen.

[0107] For example, the device model is used to determine the resolution of the projection device based on the configuration file, and call the graphics service subsystem so that the graphics service subsystem creates a virtual window according to the resolution of the projection device; sets the virtual window to transparent display to obtain a transparent virtual window, and places the transparent virtual window below the display virtual window.

[0108] For example, the device model is configured to respond to rendering instructions issued by the client virtual machine by calling the virtualized graphics processing dynamic library to parse the rendering instructions; after parsing, it submits graphics processing instructions to the graphics processor through the interface in the rendering management library, so that the graphics processor renders and synthesizes the image of the display virtual window, so that the transparent virtual window displays the same display image as the display virtual window.

[0109] For example, the device model is configured to respond to a display command issued by the client virtual machine by calling an interface in the graphics service subsystem, so that the graphics service subsystem initiates direct memory access and reads the display image from the video memory area corresponding to the transparent virtual window according to the video memory physical address.

[0110] For example, the device model is used to start the guest virtual machine so that the guest virtual machine loads the driver for the virtual graphics device interface and starts the graphics service.

[0111] The hardware device for virtualized screen projection provided in this application embodiment can execute the steps in the virtualized screen projection method provided in this application method embodiment, and has the execution steps and beneficial effects, which will not be repeated here.

[0112] Figure 3 This is a system block diagram of a hardware device provided in an embodiment of this application, such as... Figure 3As shown, the entire hardware device can contain privileged virtual machines, guest virtual machines, and a hypervisor.

[0113] The guest virtual machine can be divided into user space and kernel space. In user space, it can be divided into applications, graphics rendering APIs, open-source graphics libraries (GL driver, VK driver, gpu-virtio), and rendering management library (libdrm). In kernel space, it can contain the virtual graphics device interface (virtio-gpu).

[0114] The privileged virtual machine can be divided into user space and kernel space. In user space, it can be divided into device model (DM), graphics service subsystem, virtualized graphics processing dynamic library (Virglrenderer), and rendering management library (libdrm). In kernel space, it can contain graphics processor.

[0115] Specifically, during the startup of the guest virtual machine, the DM (DisplayPort) in the privileged virtual machine determines the DisplayPort number (a common identifier for Linux running on the privileged virtual machine, which can be obtained by calling relevant functions in the graphics service subsystem when a display is connected) based on the screen's DisplayPort number. It also obtains the screen resolution. The DM can then call the graphics service subsystem to display the guest virtual machine in full-screen mode on the physical screen, creating a virtual display window corresponding to the guest virtual machine. The DM can also call the graphics service subsystem to create a low-level transparent window (a transparent virtual window corresponding to the projection device) based on the resolution of the projection device configured in the configuration file. This window displays the content from the guest virtual machine. The projection device's window, due to its low-level and transparent properties, is not displayed on the physical screen and does not affect the guest virtual machine's display. The graphics service subsystem obtains the physical address of the transparent virtual window's video memory; this address is where each rendered frame of the image is stored. The client virtual machine calls virtio-gpu to generate rendering instructions. The DM calls the graphics processor to complete the rendering and compositing of each frame of image. Then, the client virtual machine calls virtio-gpu to generate display instructions. The DM calls the relevant interfaces of the graphics service subsystem to display the image. The graphics service subsystem uses DMA to copy each frame of image from the physical address of the video memory corresponding to the transparent virtual window to the projection device, and the projection device then projects the image onto the screen.

[0116] This architecture enables virtual screen projection, flexibly adapting to different resolutions and hardware interfaces of projection devices, while ensuring that every frame of the image remains undistorted.

[0117] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. For example... Figure 4 As shown, the electronic device 400 includes one or more processors 401 and memory 402.

[0118] The processor 401 may be a central processing unit (CPU) or other form of processing unit with data processing and / or instruction execution capabilities, and may control other components in the electronic device 400 to perform desired functions.

[0119] The memory 402 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 401 may execute the program instructions to implement the virtualized screen projection method of any embodiment of this application described above and / or other desired functions. Various contents such as initial external parameters and thresholds may also be stored in the computer-readable storage medium.

[0120] In one example, the electronic device 400 may further include an input device 403 and an output device 404, these components being interconnected via a bus system and / or other forms of connection mechanisms (not shown). The input device 403 may include, for example, a keyboard, a mouse, etc. The output device 404 may output various information to the outside, including warning messages, braking force, etc. The output device 404 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0121] Of course, for the sake of simplicity, Figure 4 Only some of the components of the electronic device 400 relevant to this application are shown in this illustration; components such as buses, input / output interfaces, etc., are omitted. In addition, the electronic device 400 may include any other suitable components depending on the specific application.

[0122] In addition to the methods and devices described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps of the virtualization projection method provided in any embodiment of this application.

[0123] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0124] Furthermore, embodiments of this application may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps of the virtualization projection method provided in any embodiment of this application.

[0125] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0126] It should be noted that the terminology used in this application is for the purpose of describing specific embodiments only and is not intended to limit the scope of this application. As shown in the specification and claims of this application, unless the context clearly indicates otherwise, words such as "a," "an," "an," and / or "the" do not specifically refer to the singular and may also include the plural. The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, or apparatus. Without further limitations, an element defined by the phrase "comprising an..." does not exclude the presence of other identical elements in the process, method, or apparatus that includes said element.

[0127] It should also be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on this application. Unless otherwise expressly specified and limited, the terms "installed," "connected," "linked," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication between two elements. For those skilled in the art, the specific meaning of the above terms in this application can be understood according to the specific circumstances.

[0128] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. The above descriptions are only preferred embodiments of this application. It should be noted that due to the limitations of written expression, while there are objectively infinite specific structures, those skilled in the art can make several improvements, modifications, or changes without departing from the principles of this application, and can also combine the above technical features in an appropriate manner. These improvements, modifications, changes, or combinations, or the direct application of the inventive concept and technical solution to other situations without modification, should all be considered within the scope of protection of this application.

Claims

1. A virtualized screen projection method, characterized in that, The method, executed by a privileged virtual machine in a hardware device, the hardware device further comprising at least one guest virtual machine, includes: Obtain the display port number of the preset screen corresponding to the client virtual machine, and determine the resolution of the preset screen based on the display port number; A virtual display window corresponding to the client virtual machine is created based on the resolution of the preset screen, and the virtual display window is displayed in the preset screen corresponding to the client virtual machine. The virtual display window is used to display the running content of the client virtual machine in full screen. The resolution of the projection device is determined based on the configuration file, and a virtual window is created according to the resolution of the projection device; Set the virtual window to transparent display to obtain a transparent virtual window, and place the transparent virtual window below the displayed virtual window; Obtain the physical address of the video memory corresponding to the transparent virtual window; In response to the rendering command issued by the client virtual machine, a display image identical to that of the display virtual window is rendered for the transparent virtual window, so that the display image is stored in the video memory area corresponding to the video memory physical address. The resolution of the transparent virtual window corresponds to the resolution of the projection device, and the resolution of the display virtual window corresponds to the resolution of the preset screen. In response to the display command issued by the client virtual machine, the display image is obtained from the video memory area corresponding to the physical address of the video memory, and the display image is sent to the projection device for projection display.

2. The method according to claim 1, characterized in that, In response to the rendering command issued by the client virtual machine, render the same display image for the transparent virtual window as the display virtual window, including: In response to the rendering command issued by the guest virtual machine, the virtualization graphics processing dynamic library is invoked to parse the rendering command; After parsing is completed, graphics processing instructions are submitted to the graphics processor through the interface in the rendering management library, so that the graphics processor renders and synthesizes the image according to the display image of the display virtual window, so that the transparent virtual window displays the same display image as the display virtual window.

3. The method according to claim 2, characterized in that, In response to a display command issued by the guest virtual machine, the display image is retrieved from the video memory region corresponding to the physical address of the video memory, including: In response to the display command issued by the client virtual machine, the interface in the graphics service subsystem is invoked to enable the graphics service subsystem to initiate direct memory access and read the display image from the video memory area corresponding to the transparent virtual window according to the video memory physical address.

4. The method according to claim 3, characterized in that, The rendering instruction and the display instruction are generated by the graphics service of the client virtual machine calling the virtual graphics device interface.

5. The method according to claim 4, characterized in that, Before creating the display virtual window corresponding to the guest virtual machine, the method further includes: Load the graphics processor driver and start the graphics service subsystem; Start the guest virtual machine so that it loads the driver for the virtual graphics device interface and starts the graphics service.

6. A hardware device for virtualized screen projection, characterized in that, The hardware device includes a privileged virtual machine and at least one guest virtual machine, wherein; The client virtual machine is used to issue rendering instructions and display instructions to the privileged virtual machine; The privileged virtual machine is used to create a display virtual window corresponding to the client virtual machine, display the display virtual window within a preset screen corresponding to the client virtual machine, create a transparent virtual window below the display virtual window, obtain the physical address of the video memory corresponding to the transparent virtual window, and, in response to rendering instructions issued by the client virtual machine, render a display image identical to the display virtual window for the transparent virtual window, so that the display image is stored in the video memory area corresponding to the physical address of the video memory. In response to a display sending instruction issued by the client virtual machine, the display image is obtained from the video memory area corresponding to the physical address of the video memory and sent to a projection device for projection display. The resolution of the transparent virtual window corresponds to the resolution of the projection device, and the resolution of the display virtual window corresponds to the resolution of the preset screen. The privileged virtual machine includes a device model and a graphics service subsystem; The device model is used to obtain the display port number of the preset screen corresponding to the client virtual machine, determine the resolution of the preset screen based on the display port number, and call the graphics service subsystem so that the graphics service subsystem creates a display virtual window corresponding to the client virtual machine based on the resolution of the preset screen. The display virtual window is used to display the running content of the client virtual machine in full screen. The device model is used to determine the resolution of the projection device based on the configuration file, and call the graphics service subsystem so that the graphics service subsystem creates a virtual window according to the resolution of the projection device; the virtual window is set to transparent display to obtain a transparent virtual window, and the transparent virtual window is placed below the display virtual window.

7. The hardware device according to claim 6, characterized in that, in; The device model is used to call the graphics service subsystem; The graphics service subsystem is used to create a display virtual window corresponding to the client virtual machine, create a transparent virtual window located below the display virtual window, and read the display image from the video memory area corresponding to the transparent virtual window according to the video memory physical address.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program or instructions that cause a computer to perform the steps of the virtualized screen projection method as described in any one of claims 1 to 5.

Images