Image processing system and method
Through the one-to-one correspondence between the rendering task creation requests and rendering instructions between the virtual graphics card and the physical machine system, the problem that multiple containers cannot perform image calculations at the same time is solved, the image processing efficiency is improved, and GPU resource waste and communication efficiency is avoided Low problem.
Patent Information
- Application Number
- CN202110497113.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-05-07
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2041-05-07
AI Technical Summary
In the prior art, multiple containers cannot perform image calculations at the same time, resulting in inefficient image processing, waste of GPU resources and inefficient communication between processes.
Using the rendering task creation request and rendering instructions between the virtual graphics card and the physical machine system, the virtual graphics card sends a rendering task creation request to the physical machine system through the virtual graphics card, and performs the rendering task in the physical machine system, and uses the rendering service module. Image rendering with the image processor.
Multiple containers can perform image calculations simultaneously, improving image processing efficiency, and avoiding the problems of waste of GPU resources and low communication efficiency between processes.
Smart Images

Figure CN113240571B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing, and in particular, to an image processing system and method. Background Art
[0002] In recent years, cloud phones have also begun to be used in specific industries. Virtual mobile phone systems are deployed on cloud servers. Users can access the cloud through the Internet to use pre-set mobile phone systems. The user interface (UserInterface) of the mobile phone system is very important. A good UI interface can make the software personalized and tasteful, and the operation of the software becomes comfortable and simple. Therefore, how to process the image data displayed on the UI interface is crucial. Generally, containers can be used to process image data. In related technologies, there are two ways to process image data using containers:
[0003] The first type: When the container processes simple image data, it uses CPU resources for calculation. When the container processes complex images, such as 3D images, the CPU's ability to process images is very limited, so the physical system's graphics processor GPU is called for processing. However, in this processing method, the container will occupy GPU resources alone, making other containers unable to use the GPU. Therefore, there is a waste of GPU resources and a technical problem that multiple containers cannot perform image calculations at the same time.
[0004] The second method: The container sends rendering instructions to the physical system space through a socket, and then uses the GPU resources of the physical machine for image calculation. Although this method of forwarding rendering instructions to the physical system space through a socket solves the problem of a certain rendering instruction occupying the GPU alone, this method still has technical problems such as multiple containers cannot perform image calculations at the same time and is affected by the quality of inter-process communication, resulting in low efficiency.
[0005] Therefore, both of the above two methods have the technical problems of being unable to satisfy the requirement of performing a large amount of image calculations on multiple containers at the same time and having low efficiency in processing image data.
[0006] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention
[0007] The embodiments of the present application provide an image processing system and method to at least solve the technical problem of low image processing efficiency caused by the inability to perform image calculations on multiple containers at the same time in the related art.
[0008] According to one aspect of an embodiment of the present application, an image processing system is provided, including: a virtual machine system, the virtual machine system including: at least one container and at least one virtual graphics card; wherein the at least one container is used to generate rendering instructions; the at least one virtual graphics card is used to send a creation request for creating a rendering task to a physical machine system; the physical machine system is used to create a rendering task according to the rendering instruction under the triggering of the creation request, and execute the rendering task to render the initial image to be rendered corresponding to the rendering task to obtain a target image.
[0009] Optionally, the physical machine system includes: a rendering service module and an image processor, which, when triggered by a creation request, creates a rendering task according to a rendering instruction, and executes the rendering task to render an initial image to be rendered to obtain a target image, including: a rendering service module, which receives a creation request, and, when triggered by the creation request, creates a rendering task according to a rendering instruction, and forwards the rendering task to the image processor; an image processor, which is used to render the initial image to be rendered corresponding to the rendering task to obtain a target image.
[0010] Optionally, forwarding the rendering task to the image processor includes: a rendering service module calling an open graphics library application program interface Opengl API to forward the rendering task to the image processor.
[0011] Optionally, the virtual machine system is further used to allocate a graphics card channel to the virtual graphics card, wherein the graphics card channel is used to send rendering instructions to the rendering service module, and the graphics card channel and the virtual graphics card are in one-to-one correspondence.
[0012] Optionally, the virtual machine system is further used to create a device file corresponding to the virtual graphics card for the virtual graphics card, and the container's operation instructions for the virtual graphics card are issued through the device file, wherein the device file and the container are in one-to-one correspondence, and the operation instructions include: rendering instructions.
[0013] Optionally, there is a one-to-one correspondence between the container and the virtual graphics card, and there is a one-to-one correspondence between the rendering instruction and the rendering task.
[0014] Optionally, the image processor includes: an image accelerator card GPU and a graphics processor CPU.
[0015] According to one aspect of an embodiment of the present application, an image processing method is also provided, including: receiving a rendering instruction and a creation request for creating a rendering task from a virtual machine system; wherein the virtual machine system includes: at least one container and at least one virtual graphics card, at least one container is used to generate a rendering instruction, and at least one virtual graphics card is used to send a creation request for creating a rendering task to a physical machine system; under the triggering of the creation request, a rendering task is created according to the rendering instruction; and the rendering task is executed to render the initial image to be rendered corresponding to the rendering task to obtain a target image.
[0016] Optionally, a rendering task is executed to render the initial image to be rendered corresponding to the rendering task to obtain a target image, including: a rendering service module, which receives a creation request, and creates a rendering task according to a rendering instruction when triggered by the creation request, and forwards the rendering task to an image processor; an image processor, which is used to render the initial image to be rendered corresponding to the rendering task to obtain a target image.
[0017] According to one aspect of an embodiment of the present application, an image processing device is also provided, including: a receiving module, used to receive rendering instructions and a creation request for creating a rendering task from a virtual machine system; wherein the virtual machine system includes: at least one container and at least one virtual graphics card, at least one container, used to generate rendering instructions, at least one virtual graphics card, used to send a creation request for creating a rendering task to a physical machine system; a creation module, used to create a rendering task according to the rendering instruction under the triggering of the creation request; an execution module, used to execute the rendering task to render the initial image to be rendered corresponding to the rendering task to obtain a target image.
[0018] According to one aspect of an embodiment of the present application, a non-volatile storage medium is further provided. The non-volatile storage medium includes a stored program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute any image processing method.
[0019] According to one aspect of an embodiment of the present application, a processor is further provided, and the processor is used to run a program, wherein any image processing method is executed when the program is run.
[0020] In an embodiment of the present application, a virtual graphics card is used to receive a creation request for creating a rendering task of the virtual graphics card and a rendering instruction of the container, and when the creation request is triggered, a rendering task is created according to the rendering instruction, and the rendering task is executed to obtain a rendered image. The technical effect of creating a rendering task according to the rendering instruction in the container and the creation request of the virtual graphics card, that is, creating a corresponding rendering task for each rendering instruction, and sending the rendering task to the physical machine system for rendering is achieved, thereby solving the technical problem of low image processing efficiency caused by the inability of multiple containers to perform image calculations at the same time in the related technology. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0022] Figure 1 is a schematic structural diagram of an optional image processing system according to an embodiment of the present application;
[0023] Figure 2 is a flowchart of an optional image processing method according to an embodiment of the present application;
[0024] Figure 3 is a schematic structural diagram of an optional image processing device according to an embodiment of the present application;
[0025] Figure 4 It is a schematic diagram of the structure of an optional physical machine system of the present application;
[0026] Figure 5 It is a flowchart of an optional processing rendering instruction in an embodiment of the present application. DETAILED DESCRIPTION
[0027] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.
[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0029] In order for those skilled in the art to better understand the embodiments of the present application, some technical terms or nouns involved in the embodiments of the present application are explained as follows:
[0030] Hypervisor, also known as virtual machine monitor (VMM), is software, firmware or hardware used to create and execute virtual machines. The computer used by the hypervisor to execute one or more virtual machines is called the host machine, and these virtual machines are called guest machines. The hypervisor provides a virtual operating platform to execute guest operating systems (guest systems) and is responsible for managing the execution phase of other guest operating systems; these guest operating systems share virtualized hardware resources.
[0031] Virtio is an abstraction layer above the devices in a paravirtualized hypervisor. Virtio is an abstraction of a set of common simulated devices in a paravirtualized hypervisor. This setting also allows the hypervisor to export a set of common simulated devices and make them available through a common application programming interface (API). With a paravirtualized hypervisor, the guest operating system can implement a set of common interfaces and use specific device simulations behind a set of back-end drivers.
[0032] Opengl API (English: Open Graphics Library, translated as: Open Graphics Library or "Open Graphics Library") is a cross-language, cross-platform application programming interface (API) for rendering 2D and 3D vector graphics. This interface consists of nearly 350 different function calls, used to draw from simple graphics bits to complex three-dimensional scenes. Another program interface system is Direct3D, which is only used on Microsoft Windows. Opengl API is often used in CAD, virtual reality, scientific visualization programs and video game development.
[0033] Application Programming Interface (API), also known as application programming interface, is the agreement for connecting different components of a software system. As the scale of software has become increasingly large in recent years, it is often necessary to divide complex systems into small components, and the design of programming interfaces is very important. In the practice of programming, the design of programming interfaces must first make the responsibilities of the software system reasonably divided. Good interface design can reduce the mutual dependence of various parts of the system, improve the cohesion of the components, and reduce the degree of coupling between the components, thereby improving the maintainability and scalability of the system.
[0034] Graphics accelerators usually appear in the form of graphics acceleration cards, which are image adapter cards that are specially used for graphics calculations in a chip-integrated manner. Since graphics accelerator cards are much faster than CPUs in calculating graphics, computers equipped with graphics accelerator cards perform well in image processing and three-dimensional games. Graphics accelerators are expansion boards that include graphics coprocessors and video adapter functions. The graphics accelerator board takes over the graphics processing work from the CPU and handles it by itself. Therefore, when Windows applications run on computers with graphics accelerator boards, the speed and processing power are significantly improved. Graphics accelerators are generally composed of hardware with an independent CPU and corresponding software, and are specially optimized for graphics and image display.
[0035] A virtual graphics processing unit (vGPU) is a component that renders graphics on a virtual desktop.
[0036] Virtual operating system simulator: QEMU is a simulated processor written by Fabrice Bellard and distributed under the GPL license. It is widely used on the GNU / Linux platform. Bochs, PearPC, etc. are similar to it, but do not have many of its features, such as high speed and cross-platform features. Through the closed-source accelerator KQEMU, QEMU can simulate to a speed close to that of a real computer. Currently, qemu versions 0.9.1 and earlier can use the kqemu accelerator. After qemu1.0, kqemu cannot be used. The qemu-kvm acceleration module is mainly used, and the acceleration effect and stability are significantly better than kqemu.
[0037] Socket is an abstraction of the endpoint for two-way communication between application processes on different hosts in the network. A socket is one end of the process communication on the network, providing a mechanism for application layer processes to exchange data using network protocols. In terms of its position, the socket is connected to the application process at the top and the network protocol stack at the bottom. It is the interface for applications to communicate through network protocols and the interface for applications to interact with network protocol roots.
[0038] Graphics Processing Unit (GPU), also known as display core, visual processor, display chip, is a microprocessor that specializes in image and graphics related computing on personal computers, workstations, game consoles and some mobile devices (such as tablets, smart phones, etc.). GPU reduces the graphics card's dependence on the CPU and performs some of the original CPU's work, especially in 3D graphics processing. The core technologies used by GPU include hardware T&L (geometry transformation and lighting processing), cubic environment material mapping and vertex blending, texture compression and bump mapping, dual texture four-pixel 256-bit rendering engine, etc., and hardware T&L technology can be said to be the hallmark of GPU.
[0039] User Interface (UI) is the medium for interaction and information exchange between the system and the user. It realizes the conversion between the internal form of information and the form acceptable to humans. The user interface is the software designed between the user and the hardware for interaction and communication. The purpose is to enable the user to operate the hardware conveniently and efficiently to achieve two-way interaction and complete the work that the hardware wants to complete. The definition of user interface is broad, including human-computer interaction and graphical user interface. User interface exists in all fields involving information exchange between humans and machines.
[0040] A container is an entity when an image is running. A container can be created, started, stopped, deleted, paused, etc. The essence of a container is a process, but unlike a process that runs directly on the host, a container process runs in its own independent namespace. Therefore, a container can have its own root file system, its own network configuration, its own process space, and even its own user ID space. The process in the container runs in an isolated environment, and it is as if it were operating in a system independent of the host. This feature makes container-encapsulated applications safer than running directly on the host. A container is an entity when an image is running. A container can be created, started, stopped, deleted, paused, etc.
[0041] According to an embodiment of the present application, an embodiment of an image processing method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0042] Figure 1 is an image processing system according to an embodiment of the present application, such as Figure 1 As shown, the image processing system comprises:
[0043] A virtual machine system 10, wherein the virtual machine system 10 includes: at least one container 100 and at least one virtual graphics card 102; wherein the at least one container 100 is used to generate a rendering instruction; and the at least one virtual graphics card 102 is used to send a creation request for creating a rendering task to the physical machine system 20;
[0044] The physical machine system 20 is used to create a rendering task according to a rendering instruction under the triggering of a creation request, and execute the rendering task to render the initial image to be rendered corresponding to the rendering task to obtain a target image.
[0045] In the image processing system, a virtual machine system 10 is provided, and the virtual machine system 10 includes: at least one container 100 and at least one virtual graphics card 102; wherein, at least one container 100 is used to generate rendering instructions; at least one virtual graphics card 102 is used to send a creation request for creating a rendering task to a physical machine system 20; the physical machine system 20 is used to create a rendering task according to the rendering instruction under the triggering of the creation request, and execute the rendering task to render the initial image to be rendered corresponding to the rendering task to obtain a target image; the rendering task is created according to the rendering instruction in the container and the creation request of the virtual graphics card, that is, the rendering task corresponding to each rendering instruction is created, and the rendering task is sent to the physical machine system for rendering, thereby solving the technical problem of low image processing efficiency caused by the inability of multiple containers to perform image calculations at the same time in the related technology.
[0046] It should be noted that the virtual machine system may be provided with multiple containers and virtual graphics cards corresponding to the multiple containers.
[0047] In some optional embodiments of the present application, the physical machine system 20 includes: a rendering service module 200 and an image processor 202. When triggered by a create request, the rendered target image can be obtained in the following manner. Specifically, a rendering task is created according to a rendering instruction, and the rendering task is executed to render the initial image to be rendered to obtain the target image. Optionally, a rendering service module can be used to receive a create request, and when triggered by the create request, a rendering task is created according to the rendering instruction, and the rendering task is forwarded to the image processor. The image processor is used to render the initial image to be rendered corresponding to the rendering task to obtain the target image.
[0048] In some embodiments of the present application, the above-mentioned physical machine system also includes an open graphics library application program interface OpenglAPI 204, and the rendering task can be forwarded to the image processor in the following manner. Specifically, the rendering service module calls the open graphics library application program interface Opengl API 204 to forward the rendering task to the image processor 202.
[0049] In some embodiments of the present application, the virtual machine system is also used to allocate a graphics card channel for the virtual graphics card. It should be noted that the graphics card channel is used to send rendering instructions to the rendering service module. For example, the generated rendering instructions are sent to the virtual graphics card VGPU corresponding to the container in the container; the virtual graphics card VGPU sends the rendering instructions to the rendering service module in the physical machine system (Host) via the virtio graphics card channel, and the graphics card channel corresponds to the virtual graphics card one by one. Specifically, QEMU can be used to allocate a virtio graphics card channel to each virtual graphics card (VGPU) after the virtual machine system is started. It should be noted that the virtio channel is a channel for message and data transmission between the virtual machine system (guest) and the physical machine system (Host). For example, for the virtual graphics card VGPU-A, the channel virtio-A is allocated. For example, the Android application A on the container A sends the graphics rendering instructions to the virtual graphics card VGPU-A corresponding to the container A through the graphics processing module; the virtual graphics card VGPU-A passes the image rendering instructions to the rendering service module of the physical machine system (Host) through virtio-A.
[0050] In some embodiments of the present application, the virtual machine system is also used to create a device file corresponding to the virtual graphics card for the virtual graphics card. The container's operating instructions for the virtual graphics card are issued through the device file. It should be noted that the device file corresponds to the container one-to-one, and the operating instructions include but are not limited to: rendering instructions. Specifically, each virtual graphics card in the virtual machine system creates a corresponding device file. It should be noted that the rendering instructions generated by the application layer in the container in the virtual machine system can be sent to the physical machine system (Host) through the device file through the virtio graphics card channel. All operations of the container on the virtual graphics card are issued through the device file, wherein one virtual graphics card can create a device file. For example, after the virtual graphics card VGPU-A of the virtual machine system (guest) is initialized, the cardA device file can be created in the / dev / directory; similarly, after the virtual graphics card VGPU-B is initialized, the cardB device file can be created in the / dev / directory.
[0051] It should be noted that there is a one-to-one correspondence between containers and virtual graphics cards, and a one-to-one correspondence between rendering instructions and rendering tasks. It is understandable that each container in the virtual machine system can establish a binding relationship with the device file corresponding to each virtual graphics card. It should be noted that one container is bound to one device file. For example, after container A is started, container A opens and locks the / dev / cardA device file.
[0052] It should be noted that the image processor 202 includes but is not limited to: an image accelerator card GPU and a graphics processor CPU.
[0053] Figure 2 is an image processing method according to an embodiment of the present application, such as Figure 2 As shown, the method comprises the following steps:
[0054] Step S102, receiving a rendering instruction and a creation request for creating a rendering task from a virtual machine system; wherein the virtual machine system includes: at least one container and at least one virtual graphics card, the at least one container is used to generate a rendering instruction, and the at least one virtual graphics card is used to send a creation request for creating a rendering task to a physical machine system;
[0055] Step S104, under the triggering of the creation request, creating a rendering task according to the rendering instruction;
[0056] Step S106 , executing the rendering task to render the initial image to be rendered corresponding to the rendering task to obtain a target image.
[0057] In the image processing method, first, a rendering instruction and a creation request for creating a rendering task can be received from a virtual machine system; wherein the virtual machine system includes: at least one container and at least one virtual graphics card, at least one container is used to generate a rendering instruction, and at least one virtual graphics card is used to send a creation request for creating a rendering task to a physical machine system; finally, under the triggering of the creation request, a rendering task is created according to the rendering instruction; the rendering task is executed to render an initial image to be rendered corresponding to the rendering task to obtain a target image, thereby achieving a technical effect of creating a rendering task according to the rendering instruction in the container and the creation request of the virtual graphics card, that is, creating a corresponding rendering task for each rendering instruction, and sending the rendering task to the physical machine system for rendering, thereby solving the technical problem of low image processing efficiency caused by the inability of multiple containers to perform image calculations at the same time in the related technology.
[0058] In some embodiments of the present application, a rendering task is executed to render an initial image to be rendered corresponding to the rendering task to obtain a target image, including: a rendering service module, which receives a creation request, and creates a rendering task according to a rendering instruction when triggered by the creation request, and forwards the rendering task to an image processor; an image processor, which is used to render the initial image to be rendered corresponding to the rendering task to obtain a target image.
[0059] Figure 3 is an image processing device according to an embodiment of the present application, such as Figure 3 As shown, the device comprises:
[0060] The receiving module 40 is used to receive a rendering instruction and a creation request for creating a rendering task from the virtual machine system; wherein the virtual machine system includes: at least one container and at least one virtual graphics card, the at least one container is used to generate a rendering instruction, and the at least one virtual graphics card is used to send a creation request for creating a rendering task to the physical machine system;
[0061] A creation module 42, used to create a rendering task according to a rendering instruction under the triggering of a creation request;
[0062] The execution module 44 is used to execute the rendering task to render the initial image to be rendered corresponding to the rendering task to obtain a target image.
[0063] In the image processing device, a receiving module 40 is used to receive rendering instructions and creation requests for creating rendering tasks from a virtual machine system; wherein the virtual machine system includes: at least one container and at least one virtual graphics card, at least one container is used to generate rendering instructions, and at least one virtual graphics card is used to send a creation request for creating a rendering task to a physical machine system; a creation module 42 is used to create a rendering task according to the rendering instruction under the triggering of the creation request; an execution module 44 is used to execute the rendering task to render the initial image to be rendered corresponding to the rendering task to obtain a target image, thereby achieving the technical effect of creating a rendering task according to the rendering instructions in the container and the creation request of the virtual graphics card, that is, creating a corresponding rendering task for each rendering instruction, and sending the rendering task to the physical machine system for rendering, thereby solving the technical problem of low image processing efficiency caused by the inability of multiple containers to perform image calculations at the same time in the related technology.
[0064] Figure 4 is a schematic diagram of an optional physical machine system of the present application, such as Figure 4 As shown, the physical machine system 20 includes:
[0065] A virtual machine system 10, the virtual machine system 10 comprising: a plurality of containers 100 and a plurality of virtual graphics cards 102; and
[0066] Rendering service module 200, image processor 202, Opengl API 204; QEMU 206 for generating virtual machine system 10, and graphics driver 208 for driving graphics card.
[0067] In the physical machine system 20, there is a virtual machine system 10, which includes: multiple containers 100 and multiple virtual graphics cards 102; wherein the container 100 is used to generate rendering instructions; the virtual graphics card 102 is used to send a creation request for creating a rendering task to the physical machine system 20; a rendering service module 200 is used to create a rendering task according to the rendering instruction under the triggering of the creation request, and call the open graphics library application program interface Opengl API 204 to forward the rendering task to the image processor 202, and the image processor 202 is used to execute the rendering task to render the initial image to be rendered corresponding to the rendering task to obtain the target image; the rendering task is created according to the rendering instruction in the container and the creation request of the virtual graphics card, that is, the rendering task corresponding to each rendering instruction is created, and the rendering task is sent to the physical machine system for rendering, thereby solving the technical problem of low image processing efficiency caused by the inability of multiple containers to perform image calculations at the same time in the related technology.
[0068] It is easy to notice that QEMU 206 is used to generate and run the virtual machine system 10 on the physical machine system 20 , and the graphics card driver 208 is used to drive the graphics card in the physical machine system 20 .
[0069] Figure 5 is a schematic diagram of an optional process for processing rendering instructions in an embodiment of the present application, such as Figure 5 As shown in the figure, the specific whole processing process is:
[0070] Step 1, after the physical machine system (Host) is started and initialized, QEMU is started; a virtual machine system (guest) is established through QEMU, and in the virtual machine system (guest), according to the preset number of deployed containers N, N virtual graphics cards (VGPUs) and N containers are configured. Optionally, the number of containers and virtual graphics cards (VGPUs) in the virtual machine system (guest) is set to be the same.
[0071] Step 2: After the virtual machine system (guest) is started, QEMU allocates a virtio graphics card channel to each virtual graphics card (VGPU). It should be noted that the virtio channel is a channel for message and data transmission between the virtual machine system (guest) and the physical machine system (host). For example, for the virtual graphics card VGPU-A, allocate channel virtio-A.
[0072] Step 3: The physical machine system (Host) starts the rendering service module; the rendering server module monitors all virtio graphics card channels. It can be understood that the rendering server module in the physical machine system (Host) can monitor the virtio graphics card channels of each virtual graphics card to obtain the rendering instructions sent by each virtual graphics card in a timely manner.
[0073] Step 4: After initialization, each virtual graphics card sends a creation request to the rendering service module through its own virtio graphics card channel. The rendering service module creates a rendering subtask for the virtual graphics card according to the creation request of the virtual graphics card. For example, the virtual graphics card VGPU-A sends a creation request to the rendering service module through the virtio-A graphics card channel. After receiving the creation request, the rendering service module creates rendering subtask 1 (Task-A) for the virtual graphics card VGPU-A.
[0074] Step 5, each virtual graphics card in the virtual machine system (guest) creates a corresponding device file. It should be noted that the rendering instructions generated by the application layer in the container in the virtual machine system (guest) can be sent to the physical machine system (Host) through the device file and the virtio graphics card channel. All operations of the container on the virtual graphics card are issued through the device file, where one virtual graphics card can create a device file. For example, after the virtual graphics card VGPU-A of the virtual machine system (guest) is initialized, the cardA device file can be created in the / dev / directory; similarly, after the virtual graphics card VGPU-B is initialized, the cardB device file can be created in the / dev / directory.
[0075] Step 6: Each container in the virtual machine system (guest) establishes a binding relationship with the device file corresponding to each virtual graphics card. It should be noted that one container corresponds to one virtual graphics card, that is, one container is bound to one device file. For example, after container A is started, the graphics processing module in container A opens and locks the / dev / cardA device file.
[0076] Step 7: The container sends the generated graphics rendering instructions to the virtual graphics card VGPU corresponding to the container; the virtual graphics card VGPU sends the graphics rendering instructions to the rendering service module in the physical machine system (Host) through the virtio graphics card channel. For example, Android application A on container A sends graphics rendering instructions to the virtual graphics card VGPU-A corresponding to container A through the graphics processing module; the virtual graphics card VGPU-A passes the image rendering instructions to the rendering service module of the physical machine system (Host) through virtio-A.
[0077] Step 8: The rendering service module in the physical machine system (Host) obtains the graphics rendering instruction on the virtio graphics card channel and sends the graphics rendering instruction to the rendering subtask corresponding to the container that generated the graphics rendering instruction. It should be noted that one container corresponds to one virtual graphics card VGPU, and one virtual graphics card VGPU creates one rendering subtask, so one container corresponds to one rendering subtask.
[0078] Step 9, the rendering subtask in the physical machine system (Host) calls the Opengl API to execute the graphics rendering instruction. Specifically, the rendering subtask calls the Opengl API to send the rendering subtask to the graphics (image) acceleration card to execute the graphics rendering instruction in the rendering subtask.
[0079] Step 10, the rendering subtask obtains the target image of the rendering result through the OpenGL function; it can be understood that after obtaining the target image, the target image can be encoded and compressed and then transmitted to the remote user.
[0080] According to one aspect of an embodiment of the present application, a non-volatile storage medium is further provided. The non-volatile storage medium includes a stored program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute any image processing method.
[0081] Specifically, the above storage medium is used to store program instructions for executing the following functions to achieve the following functions:
[0082] Receive a rendering instruction and a creation request for creating a rendering task from a virtual machine system; wherein the virtual machine system includes: at least one container and at least one virtual graphics card, at least one container is used to generate a rendering instruction, and at least one virtual graphics card is used to send a creation request for creating a rendering task to a physical machine system; under the triggering of the creation request, create a rendering task according to the rendering instruction; execute the rendering task to render the initial image to be rendered corresponding to the rendering task to obtain a target image.
[0083] According to one aspect of an embodiment of the present application, a processor is provided, and the processor is used to run a program stored in a memory, wherein any image processing method is executed when the program is run.
[0084] Specifically, the processor is used to call the program instructions in the memory to implement the following functions:
[0085] Receive a rendering instruction and a creation request for creating a rendering task from a virtual machine system; wherein the virtual machine system includes: at least one container and at least one virtual graphics card, at least one container is used to generate a rendering instruction, and at least one virtual graphics card is used to send a creation request for creating a rendering task to a physical machine system; under the triggering of the creation request, create a rendering task according to the rendering instruction; execute the rendering task to render the initial image to be rendered corresponding to the rendering task to obtain a target image.
[0086] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.
[0087] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.
[0088] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.
[0089] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0090] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0091] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the relevant technology or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc. Various media that can store program codes.
[0092] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.
Claims
1. An image processing system, characterized in that: include: A virtual machine system, wherein the virtual machine system includes: at least one container and at least one virtual graphics card; wherein the at least one container is used to generate a rendering instruction; the at least one virtual graphics card is used to send a creation request for creating a rendering task to a physical machine system; the physical machine system is used to create the rendering task according to the rendering instruction under the triggering of the creation request, and execute the rendering task to render an initial image to be rendered corresponding to the rendering task to obtain a target image, wherein the physical machine system includes: a rendering service module; Among them, the virtual machine system is also used to allocate a graphics card channel for the virtual graphics card, wherein the graphics card channel is used to send the rendering instruction to the rendering service module; the rendering service module is used to receive the creation request and, under the triggering of the creation request, create the rendering task according to the rendering instruction.
2. The system according to claim 1, characterized in that The physical machine system further includes: an image processor, which, under the triggering of the creation request, creates the rendering task according to the rendering instruction and executes the rendering task to render the initial image to be rendered to obtain a target image, including: The rendering service module is further used to forward the rendering task to the image processor; The image processor is used to render the initial image to be rendered corresponding to the rendering task to obtain a target image.
3. The system according to claim 2, characterized in that Forwarding the rendering task to the image processor, comprising: The rendering service module calls the open graphics library application program interface OpenglAPI to forward the rendering task to the image processor.
4. The system according to claim 1, characterized in that The graphics card channels correspond one to one to the virtual graphics cards.
5. The image processing system according to claim 1, characterized in that: The virtual machine system is also used to create a device file corresponding to the virtual graphics card for the virtual graphics card, and the operation instructions of the container to the virtual graphics card are issued through the device file, wherein the device file and the container are in one-to-one correspondence, and the operation instructions include: the rendering instructions.
6. The system according to claim 1, characterized in that The container and the virtual graphics card are in one-to-one correspondence, and the rendering instruction and the rendering task are in one-to-one correspondence.
7. The system according to claim 1, characterized in that The image processor includes: an image accelerator card GPU and a graphics processor CPU.
8. An image processing method, characterized in that: include: Receiving rendering instructions and creation requests for rendering tasks from the virtual machine system; The virtual machine system includes: at least one container and at least one virtual graphics card, the at least one container is used to generate the rendering instruction, and the at least one virtual graphics card is used to send a creation request for creating the rendering task to the physical machine system; Under the triggering of the creation request, creating the rendering task according to the rendering instruction; Executing the rendering task to render the initial image to be rendered corresponding to the rendering task to obtain a target image; Among them, the execution of the rendering task to render the initial image to be rendered corresponding to the rendering task to obtain the target image includes: a rendering service module, which receives the creation request and creates the rendering task according to the rendering instruction under the triggering of the creation request; the virtual machine system is also used to allocate a graphics card channel to the virtual graphics card, and the graphics card channel is used to send the rendering instruction to the rendering service module.
9. The method according to claim 8, characterized in that Executing the rendering task to render the initial image to be rendered corresponding to the rendering task to obtain a target image includes: The rendering service module is further used to forward the rendering task to the image processor; The image processor is used to render the initial image to be rendered corresponding to the rendering task to obtain a target image.
10. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored program, wherein when the program is executed, the device where the non-volatile storage medium is located is controlled to execute the image processing method according to claim 8.
