Image processing method and device, electronic equipment, storage medium and program product

Abstract

The invention provides an image processing method and device, electronic equipment, a storage medium and a program product, the method is applied to an image processing module running in target equipment, the target equipment further comprises a virtual machine, and the method comprises the steps that in response to starting of the virtual machine in the target equipment, according to a physical address corresponding to a virtual frame buffer area, the virtual frame buffer area is obtained; determining a physical storage area corresponding to the virtual frame buffer area; the physical storage area is determined as an image storage area, and the image storage area is a storage area which can be accessed by the image processing module; and under the condition that the virtual machine stores image frame data to the virtual frame buffer area, reading the image frame data from the image storage area so as to render and display the image frame data. According to the embodiment of the invention, the efficiency of displaying the image frame data can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to an image processing method and apparatus, electronic equipment, computer-readable storage medium, and computer program product. Background Technology

[0002] With the continuous development of computer technology and virtualization technology, virtual machines can be started and run on target devices in various practical business scenarios to meet the needs of various business scenarios.

[0003] In the process of displaying virtual machine image frame data, related technologies require the virtual machine image frame data to be transferred multiple times between various modules of the target device, resulting in low efficiency in displaying the image frame data. Therefore, how to improve the efficiency of displaying the image frame data has become an urgent technical problem to be solved. Summary of the Invention

[0004] This disclosure provides an image processing method and apparatus, an electronic device, a computer-readable storage medium, and a computer program product.

[0005] In a first aspect, this disclosure provides an image processing method applied to an image processing module running in a target device, the target device further including a virtual machine, the method comprising: in response to the startup of the virtual machine in the target device, determining a physical storage area corresponding to the virtual frame buffer area according to the physical address corresponding to the virtual frame buffer area; determining the physical storage area as an image storage area, wherein the image storage area is a storage area accessible by the image processing module; and, when the virtual machine stores image frame data in the virtual frame buffer area, reading the image frame data from the image storage area to render and display the image frame data.

[0006] Secondly, this disclosure provides an image processing apparatus applied to an image processing module running in a target device, the target device further including a virtual machine, the apparatus including: a first region determination module, a second region determination module, and an image reading module.

[0007] The first region determination module is used to determine the physical storage region corresponding to the virtual frame buffer region based on the physical address corresponding to the virtual frame buffer region in response to the startup of the virtual machine in the target device.

[0008] The second region determination module is used to determine the physical storage region as an image storage region, wherein the image storage region is a storage region that the image processing module can access.

[0009] The image reading module is used to read the image frame data from the image storage area when the virtual machine stores the image frame data in the virtual frame buffer area, so as to render and display the image frame data.

[0010] Thirdly, this disclosure provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores one or more computer programs executable by the at least one processor, the one or more computer programs being executed by the at least one processor to enable the at least one processor to perform the image processing method described above.

[0011] Fourthly, this disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-described image processing method.

[0012] Fifthly, this disclosure provides a computer program product comprising computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the image processing method described above.

[0013] The image processing method provided in this disclosure can, when a virtual machine is started, determine the physical storage area corresponding to the virtual frame buffer area based on the physical address corresponding to the virtual frame buffer area in the virtual machine, and determine the physical storage area as the image storage area for the image processing module for the virtual machine; subsequently, when it is detected that the virtual machine stores image frame data in the virtual frame buffer area, the image frame data can be directly read from the image storage area, reducing the number of data transfers of image frame data in the target device, thereby enabling the rapid acquisition and display of the image frame data when the virtual machine stores the image frame data in the virtual frame buffer area, improving the efficiency of displaying the image frame data.

[0014] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0015] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the embodiments of the present disclosure to explain the disclosure and do not constitute a limitation thereof. The above and other features and advantages will become more apparent to those skilled in the art from the detailed description of exemplary embodiments with reference to the accompanying drawings, in which:

[0016] Figure 1 This is a flowchart of an image processing method provided in an embodiment of the present disclosure.

[0017] Figure 2 This is a schematic diagram illustrating an image processing method provided in an embodiment of this disclosure.

[0018] Figure 3 This is a block diagram of an image processing apparatus provided in an embodiment of the present disclosure.

[0019] Figure 4 This is a block diagram of an electronic device provided in an embodiment of the present disclosure. Detailed Implementation

[0020] To enable those skilled in the art to better understand the technical solutions of this disclosure, exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments of this disclosure to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0021] Where there is no conflict, the various embodiments of this disclosure and the features thereof in the embodiments may be combined with each other.

[0022] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0023] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Words such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0024] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so defined herein.

[0025] The image processing method according to embodiments of this disclosure can be executed by an electronic device such as a terminal device or a server. The terminal device can be an in-vehicle device, user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. The method can be implemented by a processor calling computer-readable program instructions stored in memory. Alternatively, the method can be executed by a server.

[0026] The technical terms used in the image processing method of this disclosure are explained below.

[0027] The virtual machine's display window is used to display the virtual machine's operating interface and running status. This display window is the interface through which the user interacts with the virtual machine, and through this window, information such as the virtual machine system interface, program interface, and running status can be viewed.

[0028] CAD software (Computer-Aided Design software) is a technical tool that uses computer systems to assist designers in drawing, modeling, analyzing, and optimizing designs.

[0029] API (Application Programming Interface): Application programming interface.

[0030] Hypervisor is a software that creates a virtualization layer between the physical hardware and the operating system. It is used to manage and allocate physical resources to multiple virtual machines, enabling them to run independently.

[0031] KVM (Kernel-based Virtual Machine) is a virtualization technology built into the Linux operating system kernel. It uses hardware-assisted virtualization to achieve isolated operation of multiple operating systems.

[0032] QEMU (Quick Emulator) is a virtualization software primarily used to simulate computer hardware and operating system environments, supporting virtualization of various processor architectures and operating systems.

[0033] The configuration space of a virtual PCI device is a predefined and structured memory area, data structure, or data object within software (such as QEMU). This configuration space stores various device information of the virtual PCI device, including but not limited to mapping information or corresponding information between virtual addresses and physical addresses.

[0034] A graphics frame is a complete, independent unit of image used to present an image on a display. A frame is a complete screen image, containing information such as the color and brightness of all pixels on the screen. It is a static image in a sequence of images. Playing multiple frames continuously (usually 30 to 120 frames per second) creates the dynamic image (video or graphical user interface interaction) that the user sees.

[0035] VNC (Virtual Network Computing) is a remote desktop control protocol based on the RFB (Remote Framebuffer) protocol, which allows users to access and control the graphical interface of another computer over a network.

[0036] SPICE (Simple Protocol for Independent Computing Environments) is a remote desktop protocol designed specifically for virtualized environments to enhance graphical interaction capabilities within virtual machines.

[0037] PCI (Peripheral Component Interconnect) is a standard hardware bus used to connect the CPU to various peripherals, such as network cards, graphics cards, and storage controllers.

[0038] Memory map (mmap): A method that directly maps a FB memory region to the process's virtual address space, allowing threads to read and write file contents as if they were accessing ordinary memory.

[0039] OpenGL (Open Graphics Library) is a graphics application programming interface that allows programs to call the GPU for graphics rendering and is suitable for cross-platform use.

[0040] DirectX is a graphics application programming interface that allows programs to call the GPU for graphics rendering, resulting in high performance.

[0041] Vulkan is a graphics application programming interface that allows programs to call the GPU for graphics rendering. It is suitable for cross-platform use and has high performance.

[0042] Graphics stack: refers to the stacked structure of various software components involved in graphics processing, from the application to the graphics processing layer.

[0043] glTexSubImage2D is an interface in OpenGL used to update sub-regions of 2D textures. It allows developers to specify a region of a texture for data updates without reloading the entire texture image.

[0044] The SwapBuffers function is used to swap pointers between the foreground and background buffers in double-buffered rendering, synchronizing the image that has been drawn in the background buffer to the foreground.

[0045] The DirectX Present function is used to submit the rendered image from the back buffer to the screen for display.

[0046] DMA (Direct Memory Access): is a technology that bypasses the CPU to transfer data directly between memory and GPU.

[0047] With the continuous development of computer and virtualization technologies, virtual machines can be started and run on target devices in various practical business scenarios to meet the needs of different business scenarios. However, in displaying image frame data from virtual machines, related technologies require multiple data transfers between various modules of the target device, resulting in low efficiency in displaying the image frame data. For example, in open-source virtualization solutions such as QEMU / KVM, there is a significant performance bottleneck in virtual machine graphics display. Specifically, these open-source virtualization solutions typically employ a framebuffer (FB) mechanism. Graphics frames in the Guest OS (guest operating system, also known as the virtual machine operating system) running in the virtual machine need to undergo multiple memory copies. For example, graphics frames can be transferred from the Guest OS's memory to the QEMU process's memory; then, the graphics frames in the QEMU process's memory are encoded and sent to the client viewer via remote desktop protocols such as VNC or SPICE. This process consumes significant CPU resources for memory copying and data encoding / decoding. Therefore, the technical drawbacks of the aforementioned open-source virtualization solutions include: long data transmission paths and numerous replications, resulting in high screen latency and low refresh rates, making it difficult to support high-quality graphics applications (such as games, video playback, CAD design, etc.), and failing to fully utilize the hardware acceleration capabilities of the host machine's GPU.

[0048] Based on this, the present disclosure provides an image processing method and apparatus, an electronic device, a computer-readable storage medium, and a computer program product, as detailed in the following embodiments.

[0049] Figure 1 A flowchart illustrating an image processing method provided in an embodiment of this disclosure. (Refer to...) Figure 1The image processing method is applied to the target device, and the method includes steps S11-S13.

[0050] Step S11: In response to the startup of the virtual machine in the target device, determine the physical storage area corresponding to the virtual frame buffer area based on the physical address corresponding to the virtual frame buffer area.

[0051] The target device can be understood as a computing device that executes the image processing method of this disclosure. For example, the target device can be a server, a node in a distributed cluster, a client, a mobile terminal, etc., and this disclosure does not impose specific limitations on it. In a virtualization scenario, the target device can be a host machine running a virtual machine.

[0052] The virtual frame buffer area can be understood as the frame buffer area (FB) in the virtual machine. This virtual frame buffer area (also simply called the frame buffer) is used to store the image frame data of the virtual machine; the virtual frame buffer area can be the virtual machine's memory area or cache. In this embodiment, the virtual frame buffer area can be simply referred to as FB.

[0053] The physical storage area can be understood as the actual storage area in the target device. For example, the physical storage area can be the memory area, external storage area, video memory area, cache area, etc. in the target device. This disclosure does not impose specific limitations on this.

[0054] The virtual frame buffer region is obtained by virtualizing the physical storage region in the target device. The physical storage region corresponding to the virtual frame buffer region can be understood as the physical storage region used to obtain the virtual frame buffer region through virtualization.

[0055] An image processing module can be understood as a module used to process image frame data. This module can be an image processing program or an image processing process. The image processing process can be a process running on the host machine, such as a process running in the host machine's user space. An image processing program can be a program running on the host machine.

[0056] Specifically, the image processing method provided in this embodiment can be applied to an image processing module running in a target device. The target device also includes a virtual machine, which can be started and run by a hypervisor. In response to the startup of the virtual machine, the hypervisor running in the target device can allocate a virtual frame buffer area for the virtual machine. The virtual frame buffer area is obtained by virtualizing the physical storage area in the target device. In order to improve the display efficiency of the image frame data of the virtual machine, the image processing module running in the target device can determine the physical storage area corresponding to the virtual frame buffer area according to the mapping relationship between the virtual frame buffer area and the physical address, so as to facilitate the rapid acquisition of image frame data based on the physical storage area.

[0057] In some embodiments, determining the physical storage area corresponding to the virtual frame buffer region based on the physical address of the virtual frame buffer region includes: receiving the physical address corresponding to the virtual frame buffer region sent by the virtual machine management module, wherein the physical address is determined by the virtual machine management module based on the virtual address of the virtual frame buffer region and the address mapping relationship, and the virtual address is sent to the virtual machine management module by the virtual display driver running in the virtual machine; and determining the physical storage area corresponding to the virtual frame buffer region based on the physical address.

[0058] Specifically, the image processing method provided in this disclosure can use a virtual display driver running in the virtual machine to send the virtual address of the virtual frame buffer region to the virtual machine management module, wherein the virtual machine management module is a module running in the target device for managing the virtual machine; then, when the virtual address is received, the virtual machine management module determines the physical address corresponding to the virtual address according to the mapping relationship, and sends the physical address to the image processing module; finally, when the image processing module receives the physical address, it determines the physical storage region corresponding to the virtual frame buffer region according to the physical address.

[0059] The virtual display driver can be understood as a driver running in a virtual machine. This virtual display driver is used to manage the virtual frame buffer area and detect whether image frame data is written to the virtual frame buffer area; for example, the virtual display driver can be a virtual graphics card driver running in a virtual machine.

[0060] A virtual machine management module can be understood as a module used to manage the virtual machine. For example, the virtual machine management module can be a Hypervisor, KVM, or QEMU; or, the virtual machine management module can be a process, program, etc. running on the target device.

[0061] Address mapping can be understood as information used to represent the correspondence (i.e., mapping relationship) between the virtual address of the virtual frame buffer area and the physical address of the physical storage area. For example, the address mapping relationship can be an address mapping table or the configuration space of a virtual PCI device. The address mapping table can record the virtual address of the virtual frame buffer area and its corresponding physical address.

[0062] Taking the application of the image processing method provided in the embodiments of this disclosure in a graphics frame display scenario as an example, the image processing method will be explained and described. The target device can run an image processing system to improve the graphics display performance in a virtualization environment. The system includes a virtual machine process (such as an improved QEMU process), a hypervisor (such as KVM), and a virtual graphics card driver located in the Guest OS on the host machine (i.e., the target device).

[0063] The virtual graphics card driver runs in the Guest OS kernel and is responsible for managing a dedicated, fixed frame buffer (FB) and triggering an interrupt when a frame (i.e., image frame data) is updated.

[0064] The virtual machine process (i.e., the image processing module) runs in the host machine's user space and includes a memory mapping module, an interrupt handling module, and a GPU rendering module. This virtual machine process creates and binds a display window, and is responsible for rendering the image frame data in the mapped Guest frame buffer to the corresponding display window of the virtual machine through the host machine's GPU acceleration.

[0065] Based on this, in the process of efficiently displaying graphics frames in a virtualized environment, it is first necessary to start the virtual machine, start the improved virtual machine process, and create the display window corresponding to the virtual machine; then, the Guest OS in the virtual machine loads the dedicated virtual graphics card driver.

[0066] Using this dedicated virtual graphics card driver, a fixed-size frame buffer (FB) can be allocated in the Guest OS memory; and the virtual graphics card driver can inform the virtual machine process on the host machine of the physical address information of the FB through the configuration space (i.e., mapping relationship) of the virtual PCI device.

[0067] Based on the above embodiments, this disclosure proposes an efficient virtual graphics display system (i.e., the above-mentioned image processing system). This system can deploy a virtual graphics card driver in the Guest OS and modify the virtual machine process on the host machine. Subsequently, a direct and efficient graphics data transmission and rendering path from the Guest OS frame buffer to the host machine GPU can be established through the virtual graphics card driver and the virtual machine process based on memory mapping and interrupt mechanisms.

[0068] Step S12: The physical storage area is determined as the image storage area, wherein the image storage area is a storage area that the image processing module can access.

[0069] The image storage area can be an area managed by the image processing module for storing image frame data. For example, the image storage area can be the process address space, cache area, or memory area of ​​the image processing module.

[0070] Specifically, the image processing method provided in this disclosure requires determining the process address space of the image processing module after determining the physical storage area, and adding the physical storage area to the process address space of the image processing module to obtain the image storage area for the virtual machine.

[0071] In some embodiments, the image processing module includes an image processing process, and the image storage area includes the process address space of the image processing process, the process address space being used to store the image frame data; determining the physical storage area as the image storage area includes: mapping the physical storage area to the process address space of the image processing process based on memory mapping, wherein the process address space includes the physical storage area.

[0072] Specifically, the image processing method provided in this disclosure can add the physical storage area to the image storage area of ​​the image processing module for the virtual machine, and the method includes: mapping the physical storage area to the process address space of the image processing process for the virtual machine based on memory mapping.

[0073] Following the previous example, after the virtual graphics card driver informs the virtual machine process (i.e., the image processing process) on the host machine of the physical address information of the FB through the configuration space (i.e., mapping relationship) of the virtual PCI device, the virtual machine process can call the host operating system interface (such as the mmap interface) to directly map the memory area (i.e., the physical storage area) corresponding to the FB in the Guest OS to its own process address space. It should be noted that this mapping operation only needs to be completed once during the initialization phase.

[0074] As can be seen from the above embodiments, the embodiments of this disclosure can establish a direct and efficient graphics data transmission channel from the Guest OS frame buffer to the host GPU through the virtual graphics card driver and virtual machine process based on memory mapping. Subsequently, according to the interrupt mechanism, the graphics data (i.e., image frame data) in the frame buffer can be transmitted to the host GPU for rendering and display through the graphics data transmission channel, thereby reducing the data transmission of image frame data and improving the data transmission efficiency of image frame data.

[0075] In some embodiments, after determining the physical storage area, the physical storage area can be used as the image storage area for the virtual machine by the image processing module.

[0076] Step S13: The physical storage area is determined as the image storage area, wherein the image storage area is a storage area that the image processing module can access.

[0077] The image frame data can be graphical data that needs to be displayed through a display window in the virtual machine. For example, the image frame data can be game images, desktop images, program interfaces or user interfaces corresponding to applications, etc., without specific restrictions.

[0078] In some embodiments, reading the image frame data from the image storage area to render and display the image frame data includes steps one and two.

[0079] Step 1: Read the image frame data from the image storage area.

[0080] Specifically, in this embodiment, an image processing module can be used to detect the virtual frame buffer area. When it is detected that the virtual machine is storing image frame data in the virtual frame buffer area, the image processing module can read the image frame data from the image storage area. That is, the virtual machine in the target device can store image frame data in the virtual frame buffer area, and the image processing module, upon detecting that image frame data is stored in the virtual frame buffer area, can read the image frame data from the image storage area.

[0081] It should be noted that, since this disclosure defines the physical storage area corresponding to the virtual frame buffer area as the image storage area of ​​the image processing module, the image processing module can directly read the image frame data written by the virtual machine to the virtual frame buffer area from the image storage area, thereby improving the efficiency of obtaining the image frame data.

[0082] In some embodiments, reading the image frame data from the image storage area includes:

[0083] Upon receiving an interrupt signal from the virtual machine, the image frame data is read from the image storage area, wherein the image frame data is an interactive image frame generated by the target application running in the virtual machine.

[0084] The interrupt signal is used to indicate that the virtual machine stores image frame data in the virtual frame buffer area. The interrupt signal may be sent by the virtual display driver to the image processing module.

[0085] An interactive image frame can be understood as an image frame generated by the target application for human-computer interaction; for example, the interactive image frame can be a game image, an application interface, a webpage, etc.; this image frame data can be displayed to the user; and the user can input corresponding input data (such as relative displacement data, key parameters, etc.) based on the image frame data through an input device (such as a mouse, keyboard, gamepad, etc.). For example, if the target application is a game application, the image frame data can be a game image, and the input device can be a mouse device; in this case, after the game image generated by the game application is displayed to the user, the user can perform operations such as moving the mouse device or pressing mouse buttons on the game image. After receiving the input data from the mouse device, the game application can perform corresponding game operations (such as switching game screens, moving game view, etc.) based on the input data, and the game application can generate a game image corresponding to the game operation (such as the game screen image after switching, the game image after moving the game view, etc.), and then display the game image to the user.

[0086] Specifically, the image processing module of the target device in this embodiment can detect interrupt signals and read image frame data from the image storage area when it receives an interrupt signal sent by the virtual machine.

[0087] Following the example above, this disclosure provides a mechanism for graphics frame update and notification. Specifically, the mechanism includes: First, an application (such as a game or CAD software) in the Guest OS of the virtual machine performs drawing operations to obtain the application image (i.e., image frame data) corresponding to the application (i.e., the target application). Then, the Guest OS uses the graphics stack (including a dedicated virtual graphics card driver) to render the application image into the shared FB (i.e., graphics frame update operation).

[0088] When an application image is rendered to a shared FB (Functional Block), the virtual machine can send a corresponding interrupt signal to the virtual machine process to inform the virtual machine process that a new application image has been written to the FB.

[0089] When a virtual machine process receives an interrupt signal sent by the virtual machine, it can quickly read image frame data directly from its own process address space, thereby improving the speed of image frame data acquisition and thus improving the efficiency of image frame display.

[0090] In some embodiments, reading the image frame data from the image storage area upon receiving an interrupt signal from the virtual machine includes: detecting a target register in the target device; determining that an interrupt signal from the virtual display driver has been received if a virtual display driver in the virtual machine writes preset parameters to the target register; and reading the image frame data from the image storage area based on the interrupt signal.

[0091] The target register can be a virtual register in a virtual machine or a real physical register in the target device.

[0092] The preset parameters can be understood as parameters that are pre-set to indicate that the virtual machine stores image frame data in the virtual frame buffer area. For example, the preset parameter can be "0x1", which indicates that the virtual machine stores image frame data in the virtual frame buffer area.

[0093] Following the example above, this disclosure provides a mechanism for updating and notifying graphics frames, specifically including the following steps.

[0094] 1. Applications in the Guest OS (such as games and CAD software) perform drawing operations to obtain application images.

[0095] 2. The Guest OS's graphics stack (including the dedicated virtual graphics card driver) renders the application images to the shared FB.

[0096] 3. When the Guest OS's virtual graphics card driver (i.e., virtual display driver) detects an update in the FB content, it sends an update interrupt signal to the host virtual machine process (i.e., the image processing module) by writing to a specific register (i.e., the target register) of the virtual PCI device.

[0097] This disclosure provides an interrupt response and data capture mechanism executed by a virtual machine process, specifically including the following steps.

[0098] 1. The host virtual machine process checks a specific register. If it detects that "0x1" has been written into the specific register, it determines that an interrupt signal has been received. The interrupt handling module of the host virtual machine process is triggered by the update interrupt signal.

[0099] 2. After the interrupt handling module is triggered, it can directly access the mapped Guest FB memory region (i.e., physical storage region) in its process address space through the interrupt handler function, and read the latest written application image from this Guest FB memory region. Since it is memory-mapped, this access does not require any data copying, thereby improving the efficiency of image frame data acquisition.

[0100] It should be noted that, in order to improve the transmission efficiency of image frame data, the virtual machine in this embodiment can be allocated multiple virtual frame buffer areas. These multiple virtual frame buffer areas can respectively store different image frame data generated by the target application, thereby enabling the rapid acquisition of image frame data to be displayed during image display and reducing the problem of a large amount of image frame data not being transmitted in a timely manner due to a small number of virtual frame buffer areas. Based on this, since there can be multiple virtual frame buffer areas, the corresponding physical storage area can also be multiple. In this case, this disclosure can define multiple physical storage areas as image storage areas. When the virtual machine stores image frame data in the target virtual frame buffer area, image frame data is read from the image storage area to render and display the image frame data. The target virtual frame buffer area can be any one of the multiple virtual frame buffer areas. Correspondingly, reading image frame data from the image storage area includes: determining the target storage area corresponding to the target virtual frame buffer area from the image storage area, and reading image frame data from the target storage area.

[0101] In some embodiments, the interrupt signal is sent by the virtual display driver to the image processing module when it receives an image update instruction sent by the virtual machine; or the interrupt signal is sent by the virtual display driver to the image processing module when it detects that the virtual frame buffer area has been stored in the virtual frame buffer area at a preset time frequency; wherein, the virtual display driver is a driver running in the virtual machine.

[0102] Specifically, in the image processing method provided in this disclosure, a virtual display driver runs in the virtual machine; when the virtual display driver receives an image update instruction sent by the virtual machine, it can send the interrupt signal to the image processing module; or when the virtual display driver detects the virtual frame buffer area at a preset time frequency and detects that the image frame data is stored in the virtual frame buffer area, it can send the interrupt signal to the image processing module.

[0103] The image update instruction can be understood as an instruction used to indicate that the virtual machine stores image frame data into the virtual frame buffer area.

[0104] The preset time frequency can be set according to the actual application scenario. For example, the preset time frequency can be the screen refresh time frequency of the target device (i.e., the time frequency or number of times the display or screen refreshes the image per second) and the vertical refresh rate.

[0105] Continuing with the previous example, after the virtual machine's guest operating system renders the application image onto the shared frame buffer (FB), it can send an image update command to the virtual graphics card driver, indicating that the FB has been updated. Upon receiving this command, the virtual graphics card driver can determine that the FB content has been updated. Therefore, it sends an update interrupt signal to the host virtual machine process by writing to a specific register of the virtual PCI device.

[0106] Alternatively, the virtual graphics driver can detect the FB at the vertical synchronization frequency. When an update to the FB content is detected, the virtual graphics driver sends an update interrupt signal to the host virtual machine process by writing to a specific register of the virtual PCI device.

[0107] As can be seen from the above embodiments, the image processing method provided by this disclosure greatly shortens the transmission path of the graphics frame from the Guest OS frame buffer to the host display, eliminates unnecessary memory copying, and improves the transmission efficiency of image frame data.

[0108] Step 2: Render and display the image frame data using the graphics processing unit.

[0109] The graphics processing unit can be a unit configured in the target device for rendering, displaying, and other processing of image frame data; for example, the graphics processing unit can be a GPU.

[0110] In some embodiments, rendering and displaying the image frame data using a graphics processing unit includes:

[0111] The graphics processing unit is controlled to render the image frame data to the display window corresponding to the virtual machine. The display window is used to display the image frame data and is created during the startup of the virtual machine.

[0112] Following the example above, the virtual machine process in this embodiment includes a GPU rendering module. When the virtual machine process obtains image frame data, it can use the GPU rendering module to perform a rendering operation. The rendering operation refers to instructing the host GPU to render the image frame data onto a full-screen quadrilateral and present the rendered image frame data to the display window bound to the virtual machine process.

[0113] As can be seen from the above embodiments, image rendering and display operations are completed by the GPU's DMA and copy engine. This step is faster than using the CPU to copy line by line. Since the CPU is only responsible for initiating instructions, a lot of CPU resources are saved, making the CPU-side operation smoother.

[0114] In some embodiments, controlling the graphics processing unit to render the image frame data to the display window corresponding to the virtual machine includes: updating the image frame data to a preset image texture corresponding to the virtual machine to obtain a texture to be rendered; controlling the graphics processing unit to render the texture to be rendered through an image rendering instruction to obtain an image to be displayed, and displaying the image to be displayed in the display window corresponding to the virtual machine.

[0115] Image rendering instructions can be understood as instructions used to control the graphics processing unit to render and display the texture to be rendered.

[0116] Following the example above, during the startup of the virtual machine, the virtual machine process of this embodiment can initialize a host GPU graphics API context (such as DirectX, OpenGL, or Vulkan) and create a texture object (i.e., a preset image texture) that matches the size of the shared FB.

[0117] During the process of performing GPU-accelerated rendering operations using a virtual machine process, the GPU rendering module of the virtual machine process uses graphics API functions (such as OpenGL's glTexSubImage2D or the DirectX interface) to directly update the texture object with the image frame data accessed from the process address space.

[0118] Subsequently, the GPU rendering module performs rendering operations (such as OpenGL's SwapBuffers or DirectX Present), and through control instructions (i.e. image rendering instructions), controls the host GPU to render the aforementioned texture object onto a full-screen quadrilateral to obtain the image to be displayed; then, the image to be displayed is presented to the display window bound to the virtual machine process.

[0119] As can be seen from the above embodiments, the image processing method provided in this disclosure has extremely high graphics performance and low CPU overhead. The extremely high graphics performance refers to the elimination of data copying from the Guest (virtual machine) to the Host (host machine) through shared memory mapping, and the elimination of encoding / decoding overhead through GPU hardware acceleration, supporting high resolution, high refresh rates, and 3D applications. In other words, during the rendering and display of image frame data, the GPU hardware is used to encode and decode the image frame data, thereby reducing CPU overhead. The low CPU overhead refers to the fact that graphics rendering and presentation work is offloaded to the GPU, significantly reducing the CPU utilization of the host machine.

[0120] The image processing method provided in this disclosure can, when a virtual machine is started, determine the physical storage area corresponding to the virtual frame buffer area based on the physical address corresponding to the virtual frame buffer area in the virtual machine, and determine the physical storage area as the image storage area for the image processing module for the virtual machine; subsequently, when it is detected that the virtual machine stores image frame data in the virtual frame buffer area, the image frame data can be directly read from the image storage area, reducing the number of data transfers of image frame data in the target device, thereby enabling the rapid acquisition and display of the image frame data when the virtual machine stores the image frame data in the virtual frame buffer area, improving the efficiency of displaying the image frame data.

[0121] Taking the application of the image processing method provided in the embodiments of this disclosure in a graphics frame display scenario as an example, the image processing method will be explained and described. Figure 2 This is a schematic diagram illustrating an image processing method provided in an embodiment of the present disclosure, based on... Figure 2 As can be seen, the present disclosure provides an image processing system for improving graphics display performance in a virtualized environment. The system includes a virtual machine process (such as an improved QEMU process) and a hypervisor (such as KVM) located on the host machine, as well as a virtual graphics card driver located in the Guest OS.

[0122] The Hypervisor is used to create and run the virtual machine; and it is also used to perform operations such as framebuffer mapping and interrupt injection.

[0123] The Guest OS is a virtual machine operating system running within the virtual machine. The virtual graphics card driver (Framebuffer Driver) runs within the Guest OS kernel, managing a dedicated, fixed frame buffer (FB) and triggering an interrupt when a frame is updated. The Guest OS also runs an application and a mouse driver, which establishes the connection between the Guest OS and the virtual mouse device.

[0124] The host virtual machine process runs in the host user space and includes a memory mapping module, an interrupt handling module, and a GPU rendering module. This virtual machine process creates and binds a display window, responsible for rendering the image frame data from the mapped Guest frame buffer to the corresponding display window of the virtual machine using the host GPU acceleration. It should be noted that this virtual machine process also performs input capture operations, acquiring the image frame data written to the Guest frame buffer and displaying it in the virtual machine's display window.

[0125] The host machine is also equipped with GPU hardware, which can perform DMA copy and rendering operations.

[0126] Based on this, the image processing method specifically includes the following steps.

[0127] Step 1: System initialization and shared buffer establishment.

[0128] Specifically, the system initialization and shared buffer establishment are executed in the following ways.

[0129] 1. Start the virtual machine, start the improved virtual machine process, and use the virtual machine process to create the corresponding display window for the virtual machine.

[0130] 2. The Guest OS in the virtual machine loads the dedicated virtual graphics card driver.

[0131] 3. This dedicated virtual graphics card driver can allocate a fixed-size frame buffer (FB) in the Guest OS memory.

[0132] 4. This dedicated virtual graphics card driver can inform the virtual machine process on the host machine of the physical address information of the FB through the configuration space (i.e., mapping relationship) of the virtual PCI device.

[0133] 5. Virtual machine processes can call the host operating system interface (such as the mmap interface) to directly map the memory region corresponding to the FB in the Guest OS into their own process address space. It should be noted that this mapping operation only needs to be completed once during the initialization phase.

[0134] 6. The virtual machine process initializes a graphics API context for the host GPU and creates a texture object that matches the size of the shared FB.

[0135] Step 2: Graphics frame update and notification.

[0136] Specifically, the execution method for graphics frame updates and notifications is as follows.

[0137] 1. Applications (such as games and CAD software) in the Guest OS of the virtual machine perform drawing operations to obtain the application images corresponding to the applications.

[0138] 2. This Guest OS uses the graphics stack (including a dedicated virtual graphics card driver) to render application images to a shared FB (i.e., graphics frame update operation).

[0139] 3. When the dedicated virtual graphics card driver detects an update to the shared FB (Framework Area) content, it sends an update interrupt signal to the host virtual machine process by writing to a specific register of the virtual PCI device. Specifically: After the virtual machine's guest operating system renders the application image to the shared FB, it can send an image update command to the virtual graphics card driver, indicating that the FB has been updated. Upon receiving the image update command, the virtual graphics card driver can determine that an update has been detected in the FB content. Therefore, the virtual graphics card driver sends an update interrupt signal to the host virtual machine process by writing to a specific register of the virtual PCI device. Alternatively, the virtual graphics card driver can detect the FB at the vertical synchronization frequency. When an update is detected in the FB content, the virtual graphics card driver sends an update interrupt signal to the host virtual machine process by writing to a specific register of the virtual PCI device.

[0140] Step 3: Interruption response and data capture.

[0141] Specifically, the execution methods for interrupt response and data capture are as follows.

[0142] 1. The host virtual machine process checks a specific register. If it detects that "0x1" has been written into the specific register, it determines that an interrupt signal has been received. The interrupt handling module of the host virtual machine process is triggered by the update interrupt signal.

[0143] 2. After the interrupt handling module is triggered, it can directly access the mapped Guest FB memory region (i.e., physical storage region) in its process address space through the interrupt handler function, and read the latest written application image from this Guest FB memory region. Since it is memory-mapped, this access does not require any data copying, thereby improving the efficiency of image frame data acquisition.

[0144] Step 4: GPU-accelerated rendering.

[0145] Specifically, the execution method of GPU-accelerated rendering is as follows.

[0146] 1. The GPU rendering module of the virtual machine process uses graphics API functions to directly update the created texture object with the accessed image frame data.

[0147] 2. The GPU rendering module performs the rendering operation, instructing the host GPU to render the above texture onto a full-screen quadrilateral to obtain the image to be displayed; then, the image to be displayed is presented to the display window bound to the virtual machine process.

[0148] Based on the above steps, the image processing method in this embodiment provides a system and method for improving graphics display performance in a virtualized environment. It can significantly shorten the transmission path of graphics frames from the Guest OS frame buffer to the host display, eliminating unnecessary memory copying. Furthermore, by offloading graphics rendering and presentation tasks from the CPU to the host GPU, hardware acceleration is achieved, significantly reducing CPU overhead.

[0149] In other words, the image processing method in this disclosure relates to the field of computer virtualization technology. Specifically, it provides a system and method for improving the graphics display performance of virtual machines based on virtualization platforms such as KVM (Kernel-based Virtual Machine). This system and method can set up a dedicated driver in the Guest OS, allocate a frame buffer, and expose its physical address to the host machine through the virtual device configuration space. The system and method provide a mechanism for the host virtual machine process to directly map the Guest's frame buffer memory to its own process space; and it provides a mechanism to trigger an interrupt through a dedicated driver to notify the host machine of frame updates, replacing continuous polling. Through this system and method, the host process can use the local GPU API to accelerate the rendering of mapped frame data as textures.

[0150] Therefore, the image processing method provided in this disclosure has extremely high performance because it employs shared memory mapping and GPU hardware acceleration techniques, avoiding multiple memory copies and software encoding / decoding, thus achieving near-physical machine graphics performance. Furthermore, it features low CPU usage because it offloads rendering tasks to the host GPU, significantly reducing the CPU's computational burden.

[0151] It is understood that the various method embodiments mentioned above in this disclosure can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this disclosure will not elaborate further. Those skilled in the art will understand that in the above methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.

[0152] In addition, this disclosure also provides an image processing apparatus, an electronic device, and a computer-readable storage medium, all of which can be used to implement any of the image processing methods provided in this disclosure. The corresponding technical solutions and descriptions are described in the corresponding section of the method and will not be repeated here.

[0153] Figure 3 This is a block diagram of an image processing apparatus provided in an embodiment of the present disclosure.

[0154] Reference Figure 3 This disclosure provides an image processing apparatus for use in a target device. The image processing apparatus includes a first region determination module 301, a second region determination module 302, and an image reading module 303.

[0155] The first region determination module 301 is used to determine the physical storage region corresponding to the virtual frame buffer region based on the physical address corresponding to the virtual frame buffer region in response to the startup of the virtual machine in the target device.

[0156] The second region determination module 302 is used to determine the physical storage region as an image storage region, wherein the image storage region is a storage region that the image processing module can access.

[0157] The image reading module 303 is used to read the image frame data from the image storage area when the virtual machine stores the image frame data in the virtual frame buffer area, so as to render and display the image frame data.

[0158] In some embodiments, the first region determination module 301 is configured to: receive the physical address corresponding to the virtual frame buffer region sent by the virtual machine management module, wherein the physical address is determined by the virtual machine management module based on the virtual address of the virtual frame buffer region and the address mapping relationship, and the virtual address is sent to the virtual machine management module by the virtual display driver running in the virtual machine; and determine the physical storage region corresponding to the virtual frame buffer region based on the physical address.

[0159] In some embodiments, the image processing module includes an image processing process, and the image storage area includes the process address space of the image processing process, the process address space being used to store the image frame data; the second region determination module 302 is used to: map the physical storage area to the process address space of the image processing process based on memory mapping, wherein the process address space includes the physical storage area.

[0160] In some embodiments, the image reading module 303 is configured to: read the image frame data from the image storage area; and render and display the image frame data using a graphics processing unit.

[0161] In some embodiments, the image reading module 303 is configured to: read the image frame data from the image storage area upon receiving an interrupt signal sent by the virtual machine, wherein the image frame data is an interactive image frame generated by a target application running in the virtual machine.

[0162] In some embodiments, the image reading module 303 is configured to: detect a target register in the target device; determine that an interrupt signal sent by the virtual display driver has been received when the virtual display driver in the virtual machine writes preset parameters to the target register; and read the image frame data from the image storage area based on the interrupt signal.

[0163] In some embodiments, the interrupt signal is sent by the virtual display driver to the image processing module when it receives an image update instruction sent by the virtual machine; or the interrupt signal is sent by the virtual display driver to the image processing module when it detects that the virtual frame buffer area has been stored in the virtual frame buffer area at a preset time frequency; wherein, the virtual display driver is a driver running in the virtual machine.

[0164] In some embodiments, the image reading module 303 is configured to: control the graphics processing unit to render the image frame data to the display window corresponding to the virtual machine, wherein the display window is used to display the image frame data and is created during the startup of the virtual machine.

[0165] In some embodiments, the image reading module 303 is configured to: update the image frame data to a preset image texture corresponding to the virtual machine to obtain a texture to be rendered; control the graphics processing unit to render the texture to be rendered through an image rendering instruction to obtain an image to be displayed, and display the image to be displayed in a display window corresponding to the virtual machine.

[0166] The image processing apparatus provided in this disclosure can, when a virtual machine is started, determine the physical storage area corresponding to the virtual frame buffer area based on the physical address corresponding to the virtual frame buffer area in the virtual machine, and determine the physical storage area as the image storage area for the image processing module for the virtual machine; subsequently, when it is detected that the virtual machine stores image frame data in the virtual frame buffer area, the image frame data can be directly read from the image storage area, reducing the number of data transfers of image frame data in the target device, thereby enabling the rapid acquisition and display of the image frame data when the virtual machine stores the image frame data in the virtual frame buffer area, improving the efficiency of displaying the image frame data.

[0167] Figure 4 This is a block diagram of an electronic device provided in an embodiment of the present disclosure.

[0168] Reference Figure 4 This disclosure provides an electronic device, which includes: at least one processor 401; at least one memory 402; and one or more I / O interfaces 403 connected between the processor 401 and the memory 402; wherein the memory 402 stores one or more computer programs that can be executed by the at least one processor 401, and the one or more computer programs are executed by the at least one processor 401 to enable the at least one processor 401 to perform the above-described image processing method.

[0169] This disclosure also provides a computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the image processing method described above. The computer-readable storage medium may be volatile or non-volatile.

[0170] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the above-described image processing method.

[0171] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0172] As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable program instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable program instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0173] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0174] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0175] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0176] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0177] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0178] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0179] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0180] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in connection with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in connection with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this disclosure as set forth by the appended claims.

Claims

1. An image processing method, characterized in that, An image processing module running in a target device, the target device further comprising a virtual machine, the method comprising: In response to the startup of the virtual machine in the target device, the physical storage area corresponding to the virtual frame buffer area is determined according to the physical address corresponding to the virtual frame buffer area; The physical storage area is defined as the image storage area, wherein the image storage area is a storage area that the image processing module can access; When the virtual machine stores image frame data in the virtual frame buffer area, the image frame data is read from the image storage area to render and display the image frame data.

2. The method according to claim 1, characterized in that, The step of determining the physical storage area corresponding to the virtual frame buffer area based on the physical address corresponding to the virtual frame buffer area includes: The virtual machine management module receives the physical address corresponding to the virtual frame buffer area sent by the virtual machine management module. The physical address is determined by the virtual machine management module based on the virtual address of the virtual frame buffer area and the address mapping relationship. The virtual address is sent to the virtual machine management module by the virtual display driver running in the virtual machine. Based on the physical address, determine the physical storage area corresponding to the virtual frame buffer area.

3. The method according to claim 1, characterized in that, The image processing module includes an image processing process, and the image storage area includes the process address space of the image processing process, which is used to store the image frame data. The step of determining the physical storage area as the image storage area includes: The physical storage area is mapped to the process address space of the image processing process using a memory mapping method, wherein the process address space includes the physical storage area.

4. The method according to claim 1, characterized in that, The step of reading the image frame data from the image storage area, rendering and displaying the image frame data includes: Read the image frame data from the image storage area; The image frame data is rendered and displayed using a graphics processing unit.

5. The method according to claim 4, characterized in that, The step of reading the image frame data from the image storage area includes: Upon receiving an interrupt signal from the virtual machine, the image frame data is read from the image storage area, wherein the image frame data is an interactive image frame generated by the target application running in the virtual machine.

6. The method according to claim 5, characterized in that, The step of reading the image frame data from the image storage area upon receiving an interrupt signal from the virtual machine includes: The target register in the target device is detected; If the virtual display driver in the virtual machine is detected writing preset parameters to the target register, it is determined that an interrupt signal sent by the virtual display driver has been received; The image frame data is read from the image storage area based on the interrupt signal.

7. The method according to claim 5 or 6, characterized in that, The interrupt signal is sent by the virtual display driver to the image processing module upon receiving an image update command from the virtual machine; or The interrupt signal is sent by the virtual display driver to the image processing module when the virtual frame buffer area is detected at a preset time frequency and the image frame data is detected to be stored in the virtual frame buffer area; The virtual display driver is a driver that runs in the virtual machine.

8. The method according to claim 4, characterized in that, The rendering and display of the image frame data using the graphics processing unit includes: The graphics processing unit is controlled to render the image frame data to the display window corresponding to the virtual machine. The display window is used to display the image frame data and is created during the startup of the virtual machine.

9. The method according to claim 8, characterized in that, The step of controlling the graphics processing unit to render the image frame data to the display window corresponding to the virtual machine includes: The image frame data is updated to the preset image texture corresponding to the virtual machine to obtain the texture to be rendered; The graphics processing unit is controlled to render the texture to be rendered by the image rendering instructions to obtain the image to be displayed, and the image to be displayed is displayed in the display window corresponding to the virtual machine.

10. An image processing apparatus, characterized in that, An image processing module applied to a target device running in a virtual machine, the device comprising: The first region determination module is used to determine the physical storage region corresponding to the virtual frame buffer region based on the physical address corresponding to the virtual frame buffer region in response to the startup of the virtual machine in the target device. The second region determination module is used to determine the physical storage region as an image storage region, wherein the image storage region is a storage region that the image processing module can access; The image reading module is used to read the image frame data from the image storage area when the virtual machine stores the image frame data in the virtual frame buffer area, so as to render and display the image frame data.

11. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, the one or more computer programs being executed by the at least one processor to enable the at least one processor to perform the method as described in any one of claims 1-9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program, when executed by a processor, implements the method as described in any one of claims 1-9.

13. A computer program product, characterized in that, Includes computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the method as described in any one of claims 1-9.

Images