Task processing method and system, electronic equipment and storage medium

Abstract

The invention provides a task processing method and system, electronic equipment and a storage medium, and relates to the technical field of computer virtualization. The system comprises a virtual GPU device located on a host machine, and a target application and a virtual GPU drive program deployed on a client virtual machine, wherein the virtual GPU drive program comprises a user mode drive and a kernel mode drive; a plurality of processors are arranged in the user mode driver; an adapter is arranged in the virtual GPU equipment; the target processor sends a target command to the adapter through a kernel mode driver; a resource scheduling module in the adapter is used for distributing the target command to a target scheduling unit under the module; the target scheduling unit is used for distributing physical GPU resource available information and / or CPU resource available information according to the command demand information; and the physical GPU equipment processes the target task to obtain a processing feedback result. According to the method, the resource allocation can be flexibly carried out, and the resource utilization rate of the physical GPU equipment is improved while the timeliness of processing the feedback result is improved.

Description

Technical Field

[0001] The present application relates to the technical field of computer virtualization, and in particular to a task processing method, system, electronic device and storage medium. Background Art

[0002] A virtual graphics processing unit (GPU) refers to a GPU device implemented by virtualization technology, which provides GPU-related functions to virtual machines, such as 3D rendering, audio and video encoding and decoding, and parallel computing acceleration.

[0003] Existing GPU virtualization technology is generally provided directly by GPU manufacturers. This technology uses the method of logical segmentation of physical resources to divide physical GPU devices into several virtual GPU devices, and is supported by its own closed-source GPU driver software. This type of technology cannot adapt well to the needs of elastic resource allocation in cloud computing. It is divided into a fixed number and cannot be changed dynamically. In addition, virtual machine migration can only be performed between computing nodes using the same model of GPU, which seriously limits the reasonable distribution of computing resources.

[0004] Therefore, the existing technology cannot perform flexible physical GPU resource allocation, and the physical GPU resource utilization rate is low, which leads to poor timeliness of task processing. Summary of the invention

[0005] The present application provides a task processing method, system, electronic device and storage medium to solve the problem of poor task processing timeliness caused by low utilization rate of physical GPU resources in the prior art.

[0006] According to a first aspect of the present application, a task processing method is provided, comprising:

[0007] A target processor set in a user-mode driver of a client virtual machine generates a target command in response to a processing request of a target task, and sends the target command to an adapter set in a virtual GPU device through a kernel-mode driver set on the client virtual machine; wherein the target processor is a processor among a plurality of processors that matches the target application; the target task is a GPU task initiated by the target application, the target command is a command corresponding to the target task, and the target command contains physical GPU resource requirement information and / or CPU resource requirement information required for processing the target task;

[0008] Allocating the target command to a target scheduling unit through a resource scheduling module; wherein the target scheduling unit is a scheduling unit that matches the target application among multiple scheduling units;

[0009] Parsing command requirement information from the target command through the target scheduling unit, and allocating physical GPU resource availability information and / or CPU resource availability information according to the command requirement information;

[0010] When the physical GPU resource available information and / or the CPU resource available information are allocated, the physical GPU device and / or the CPU device processes the target task to obtain a processing feedback result.

[0011] Optionally, the target application includes: at least one of a rendering application, a video application, and a parallel computing application;

[0012] The target processor includes: at least one of a rendering processor, a video processor, and a parallel computing processor;

[0013] The target scheduling unit includes: at least one of a rendering scheduling unit, a video scheduling unit and a parallel computing scheduling unit;

[0014] The target command includes at least one of a rendering command, a video parameter and a calculation instruction.

[0015] Optionally, generating a target command by a target processor in response to a processing request of a target task, and sending the target command to the adapter by a kernel-mode driver comprises at least one of the following:

[0016] Generate the rendering command by the rendering processor in response to the processing request of the rendering task initiated by the rendering application, and send the rendering command to the adapter by the kernel state driver;

[0017] Generate the video parameters by the video processor in response to the processing request of the video task initiated by the video application, and send the video parameters to the adapter by the kernel state driver;

[0018] The parallel computing processor generates the computing instruction in response to the processing request of the parallel computing task initiated by the parallel computing application, and sends the computing instruction to the adapter through the kernel mode driver.

[0019] Optionally, the parsing command requirement information from the target command by the target scheduling unit includes at least one of the following:

[0020] Parsing rendering requirement information of the rendering task from the rendering command through the rendering scheduling unit;

[0021] parsing video requirement information of the video task from the video parameters by the video scheduling unit;

[0022] The parallel computing scheduling unit parses the computing instruction to obtain computing requirement information of the parallel computing task.

[0023] Optionally, the method further comprises:

[0024] The rendering processor converts a target task initiated by the rendering application in the first system environment into a target executable file in the second system environment; wherein the first system environment is different from the second system environment, and the physical GPU device supports the second system environment.

[0025] Optionally, the kernel-mode driver includes a command buffer and a data buffer; the method further includes:

[0026] Cache the target command obtained from the target processor through the command buffer area, and schedule a queue of commands to be processed; wherein the queue of commands to be processed is composed of the cached target commands;

[0027] The data of the target task is cached through the data cache area.

[0028] Optionally, the adapter further comprises: a command cache module and a data cache module;

[0029] The method further comprises:

[0030] caching the target command obtained from the command buffer area by the command buffer module;

[0031] The data of the target task acquired from the data cache area and the processing feedback result of the target task are cached by the data cache module.

[0032] Optionally, the virtual GPU device is further configured to perform any of the following:

[0033] Allocating physical GPU resources; wherein the physical GPU device corresponding to the virtual GPU device has a memory vram for managing the allocation of physical GPU resources;

[0034] After command parsing, rendering and / or computing resources are allocated according to the resource requirements of the target task corresponding to the target command, wherein the resources include physical GPU resources and virtual GPU resources;

[0035] Processing a command list in the command cache module, wherein the command list is a command stream in the command cache module and is used to cache rendering commands, video parameters, and calculation instructions;

[0036] Processing frame synchronization information, wherein the frame synchronization information is used to implement video acceleration processing and / or 3D rendering processing;

[0037] Managing the state of target processing operations including rendering, parallel computing, and video processing;

[0038] Managing the physical GPU resources.

[0039] According to a second aspect of the present application, a task processing system is provided; the task processing system comprises: a virtual GPU device located on a host machine, and a target application and a virtual GPU driver deployed on a client virtual machine, wherein the virtual GPU driver virtual program comprises a user state driver and a kernel state driver; wherein the user state driver is provided with a plurality of processors; the virtual GPU device is provided with an adapter, wherein the adapter comprises a resource scheduling module, and the resource scheduling module comprises a plurality of scheduling units;

[0040] a target processor, configured to generate a target command in response to a processing request of a target task, and send the target command to the adapter through the kernel-mode driver; wherein the target processor is a processor among the multiple processors that matches the target application; the target task is a GPU task initiated by the target application, the target command is a command corresponding to the target task, and the target command contains physical GPU resource requirement information and / or CPU resource requirement information required for processing the target task;

[0041] The resource scheduling module is used to allocate the target command to a target scheduling unit; wherein the target scheduling unit is a scheduling unit that matches the target application among the multiple scheduling units;

[0042] The target scheduling unit is used to parse command requirement information from the target command, and allocate physical GPU resource availability information and / or CPU resource availability information according to the command requirement information;

[0043] When the physical GPU resource available information is allocated, the physical GPU device processes the target task to obtain a processing feedback result.

[0044] According to a third aspect of the present application, there is provided an electronic device, comprising: at least one processor and a memory;

[0045] The memory stores computer-executable instructions;

[0046] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the task processing method described in the first aspect above.

[0047] According to a fourth aspect of the present application, a computer-readable storage medium is provided, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the task processing method described in the first aspect above.

[0048] According to a fifth aspect of the present application, a computer program product is provided, comprising a computer program, which implements the task processing method described in the first aspect when executed by a processor.

[0049] The present application provides a task processing method, comprising: generating a target command by a target processor in response to a processing request of a target task, and sending the target command to an adapter by a kernel-mode driver; wherein the target processor is a processor that matches a target application among multiple processors; the target task is a GPU task initiated by a target application, and the target command is a command corresponding to the target task; allocating the target command to a target scheduling unit by a resource scheduling module; wherein the target scheduling unit is a scheduling unit that matches the target application among multiple scheduling units; parsing command requirement information from the target command by the target scheduling unit, and allocating physical GPU resource available information and / or CPU resource available information according to the command requirement information; when the physical GPU resource available information and / or CPU resource available information is allocated, processing the target task by a physical GPU device and / or a CPU device to obtain a processing feedback result.

[0050] This application can achieve abstract management of physical GPU resources through the target processor set in the user-mode driver of the client virtual machine and the adapter set in the virtual GPU device, and support the free allocation and dynamic scaling of physical GPU resources. It can flexibly allocate resources, improve the timeliness of processing feedback results, and improve the resource utilization rate of the physical GPU device.

[0051] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0053] Figure 1 It is a schematic diagram of the rendering process in the existing operating system;

[0054] Figure 2 It is a flowchart of existing Windows graphics operations;

[0055] Figure 3 It is the existing Linux graphics operation flow chart;

[0056] Figure 4 A first schematic diagram of the first existing virtual GPU implementation method;

[0057] Figure 5 A second schematic diagram of the first existing virtual GPU implementation method;

[0058] Figure 6 A schematic diagram of a second existing virtual GPU implementation method;

[0059] Figure 7 A schematic diagram of a third existing virtual GPU implementation method;

[0060] Figure 8 A schematic diagram of the structure of a task processing system provided in an embodiment of the present application;

[0061] Fig. 9 A schematic diagram of a processing flow of a task processing system provided in an embodiment of the present application;

[0062] Fig.10 Another processing flow diagram of the task processing system provided in the embodiment of the present application;

[0063] Fig.11 A schematic diagram of the process between the calling interfaces, scheduling units, and virtual GPU devices provided in the embodiments of the present application;

[0064] Fig.12 A schematic diagram of the scheduling process provided in an embodiment of the present application;

[0065] Fig.13 A schematic diagram of the structure of a virtual GPU architecture provided in an embodiment of the present application;

[0066] Fig.14 A flowchart of a task processing method provided in an embodiment of the present application;

[0067] Fig.15 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0068] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0069] Here, exemplary embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application.

[0070] To facilitate understanding, the application scenarios of the embodiments of the present application are first introduced.

[0071] This embodiment analyzes computer virtualization technology as follows: With the help of the virtual machine monitor Hypervisor in virtualization technology, users can create multiple simulation environments or dedicated resources based on a single physical hardware system. The Hypervisor can be directly connected to the physical hardware system, thereby dividing a physical hardware system into different, separate security environments, namely virtual machines (VMs). The Hypervisor can separate computer resources from hardware and allocate resources appropriately, which is very important for virtual machines. Generally, physical hardware equipped with a virtual machine monitor is called a host, and virtual machines using its resources are called client virtual machines, virtual clients, etc. These client virtual machines regard computing resources (such as CPU, memory, and storage) as a set of resources that can be reallocated. Operators can control virtual instances of CPU, memory, storage, and other resources so that client virtual machines can receive the required resources when needed.

[0072] This embodiment analyzes the current status of GPU virtualization technology as follows: Virtual GPU refers to the use of virtualization technology to implement physical GPU devices, thereby providing GPU-related functions to virtual machines, such as 3D rendering, audio and video encoding and decoding, parallel computing acceleration, etc. This GPU virtualization technology is generally provided directly by GPU manufacturers. This technology uses a method of logical segmentation of physical resources to divide the physical GPU device into several virtual GPU devices, and its own closed-source GPU driver software provides functional support. This type of technology cannot adapt well to the needs of elastic resource allocation in cloud computing. It is divided into a fixed number and cannot be changed dynamically, and virtual machine migration can only be performed between computing nodes using the same model of GPU, which severely limits the reasonable distribution of computing resources. It is explained as follows in conjunction with the cloud desktop field and cloud computing field:

[0073] In the field of cloud desktops, all virtual desktop infrastructure (VDI) solutions that support GPUs use the above-mentioned GPU virtualization solution based on hardware segmentation. It is relatively expensive and limits the way physical GPU resources are allocated. Cloud desktop vendors cannot customize more reasonable management strategies based on this.

[0074] In the field of cloud computing, CPU virtualization is already relatively complete, and the infrastructure platform based on the independent virtualization solution simulator (Quick Emulator, QEMU) / open source virtual machine software (Kernel-based Virtual Machine, KVM) has been widely adopted, which can achieve elastic management of computing resources in ultra-large-scale computer clusters. However, due to the closed nature of GPU technology, virtual GPUs have not formed a unified technical route and have not been widely adopted.

[0075] Some GPU vendors' virtual GPU devices are bound to physical GPU devices, which is completely based on hardware logic and cannot be integrated with the flexible and elastic cloud computing environment. In the cloud environment, there is a demand for 3D rendering, such as film and animation production, and there is also a demand for audio and video encoding and decoding, such as live video services, and parallel computing needs, which come from AI services. In the field of cloud desktops, the need for virtual GPUs is particularly strong. Graphical desktop operations must rely on the support of various GPU functions to run smoothly, and the usual vGPU is relatively expensive and more difficult to achieve large-scale cloud deployment.

[0076] Below, this application describes the use background of the existing GPU operating system software as follows: Taking the software running process of graphics rendering as an example:

[0077] Figure 1 It is a schematic diagram of the rendering process in the existing operating system, such as Figure 1 As shown, applications using graphics rendering need to call related rendering call interfaces (Application Programming Interface, API), such as: OpenGL applications, Vulkan applications, Direct3D applications, etc. Such APIs are in turn supported by dedicated GPU drivers. In Windows and Linux systems, the GPU driver framework is mainly composed of user-mode drivers (User-Mode Driver, UMD) and kernel-mode drivers (Kernel Mode Driver, KMD). UMD is responsible for rendering resource management and computing resource allocation, and KMD is responsible for directly controlling GPU devices.

[0078] Figure 2 This is the existing Windows graphics operation flow chart. In Windows system, the GPU driver framework is called wddm, UMD mainly implements the requirements of Direct3D applications, KMD is the port miniport driver requirement, and the graphics subsystem dxgk in Windows system completes tasks such as rendering task scheduling and inter-process communication.

[0079] Figure 3This is the existing Linux graphics operation flow chart. Figure 3 As shown, in the Linux system, UDM is mainly used to implement support for OpenGL, which is supported by mesa. KDM is called kernel drm in the Linux system, and the display part together with mesa and drm is collectively called dri.

[0080] The existing solutions provide three methods to implement GPU virtualization technology:

[0081] Figure 4 The first schematic diagram of the first existing virtual GPU implementation method is a GPU virtualization technology based on hardware segmentation. Taking the AMD vGPU solution as an example, the virtual GPU of this solution is provided by directly segmenting resources of the physical GPU device. Specifically, the support parameters of the physical GPU device for the virtual GPU device are set on the host; AMD provides a vGPU driver running in the virtual machine; and the virtual GPU driver directly controls and manages the virtual GPU device provided by the physical GPU.

[0082] Figure 5 The second schematic diagram of the first existing virtual GPU implementation method is a virtual GPU implementation method based on hardware implementation. The method is applied to the GPU main management layer loaded in the host kernel. The method includes: dividing the shared resources of the graphics subsystem into multiple resource areas, in the host kernel startup stage, allocating a resource area for the kernel GPU driver layer loaded in the host kernel, in each virtual machine kernel startup stage, respectively allocating a resource area for the kernel GPU driver layer loaded in each virtual machine kernel, obtaining and sequentially forwarding the service request sent by the sender and receiving and returning the service response of the GPU, the sender includes the host and / or virtual machine. In this technical solution, the service request of the sender (virtual machine and / or host) only needs to be forwarded through the GPU main management layer loaded in the kernel, without the need to be copied and called through multiple intermediate layers.

[0083] The existing first virtual GPU implementation method has two problems. The first is that it is not flexible enough and cannot dynamically change the allocation of physical GPU resources, that is, it cannot be flexibly deployed. The second is that the price is usually expensive.

[0084] Figure 6 FIG. 1 is a schematic diagram of the second existing virtual GPU implementation method. Figure 6As shown in the figure, the second existing virtual GPU implementation method is based on API forwarding technology. The key to this method is to implement the opengl wrapper, that is, to connect the Linux in the virtual machine and the OpenGL on the Linux on the host through a remote procedure call (rpc), and the OpenGL on the host actually executes the rendering command. In the Linux virtual machine, the application calls the OpenGL API, and the OpenGL runtime actually implements the API function, and then enters the OpenGL Wrapper, which calls the OpenGL API on the host through a remote procedure call rpc. After the host enters the OpenGL runtime, it parses the OpenGL parameters, processes the relevant data, and submits it to mesa. After entering the user-mode driver (mesa), the user-mode driver (mesa) actually processes all OpenGL rendering requests and generates rendering commands and data. The kernel-mode driver (drm) reads and processes the user-mode driver data and submits rendering commands to the GPU.

[0085] Figure 7 FIG. 1 is a schematic diagram of the third existing virtual GPU implementation method. Figure 7 As shown in the figure, the third existing virtual GPU implementation method is the virtio-gpu solution. This method is implemented based on the Linux system and works on Linux. The key to its implementation lies in: user-mode driver and vGPU rendering function. Specifically, this solution implements the vGPU virtual device working logic on the host, and independently implements rendering-related functions to become a renderer; modifies the user-mode and kernel-mode drivers in the Linux system running in the virtual machine to add support for vGPU; and implements the actual functional logic of OpenGL in the renderer.

[0086] In summary, the prior art has the following problems:

[0087] (1) Existing technologies cannot fully adapt to the GPU functional requirements of virtual machines.

[0088] (2) The application interface usually directly instructs the physical GPU device to execute GPU functions. Therefore, the call of GPU functions must be directly bound to the physical GPU device of a specific manufacturer. It is impossible to adaptively use physical GPU devices of different manufacturers, and thus cannot support the adaptation of physical GPU devices of multiple manufacturers, which ultimately leads to the inability to flexibly implement the allocation of physical GPU resources.

[0089] (3) The existing technology can only provide support for 3D rendering applications on certain operating systems, but not for certain applications. For example, it cannot provide support for Direct3D applications.

[0090] (4) Some virtual GPU implementation solutions, such as the existing solution 1 mentioned above, are bound to physical GPU devices and provided by GPU manufacturers. However, they cannot technically support flexible resource allocation customization requirements and are difficult to adapt to the changing cloud computing environment.

[0091] (5) Some virtual GPU implementation solutions, such as remotegl and wiregl, achieve some of the functions provided by the physical GPU by intrusively modifying driver functions or by hooking various applications through the software of the Windows message processing mechanism. Such solutions cannot fully adapt to the functional requirements of the virtual machine guest system.

[0092] (6) Some virtual GPU implementations, such as the existing third method virtio-gpu, can only be supported by the 3D rendering API on certain operating systems.

[0093] (7) Rendering standards are often different in different operating systems. For example, Direct3D applications can run on Windows, but cannot run properly on Linux. Currently, there is no relatively complete rendering conversion solution.

[0094] In order to solve at least one of the above technical problems, the present application provides a task processing system applied to the field of computer virtualization technology for improving resource utilization of physical GPU devices.

[0095] It should be noted that this embodiment belongs to the virtual GPU technology in the field of computer virtualization technology. By designing the GPU driver in the operating system and cooperating with the virtual GPU device, the full virtualization of the physical GPU device is realized, and the functions of all physical GPU devices are supported in the client virtual machine, including 3D rendering, video codec acceleration, parallel computing acceleration, etc.

[0096] The technical solution for implementing the virtual GPU driver and virtual GPU device provided in this embodiment can be used in, but not limited to, cloud computing clusters, cloud desktops, and virtual machines in personal computers (PCs). It can be deployed in scenarios such as higher education, general education, and finance, as well as various scenarios that require computing acceleration functions such as 3D rendering and video acceleration supported by GPUs.

[0097] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0098] Embodiment 1:

[0099] Figure 8 This is a schematic diagram of the structure of a task processing system provided in an embodiment of the present application. Figure 8 As shown, the task processing system includes: a virtual GPU device 2 located on a host machine, and a target application 3 and a virtual GPU driver deployed on a client virtual machine. The virtual GPU driver includes a user-mode driver 4 and a kernel-mode driver 5.

[0100] The user-mode driver 4 is provided with a plurality of processors; the virtual GPU device 2 is provided with an adapter 21 , the adapter 21 includes a resource scheduling module 211 , and the resource scheduling module 211 includes a plurality of scheduling units.

[0101] It should be understood that the host machine is referred to as the host, the client virtual machine is referred to as the guest, the virtual GPU device 2 is referred to as the vGPU device, the user state driver 4 is referred to as the user layer driver, and the kernel state driver 5 is referred to as the kernel layer driver.

[0102] The target processor 41 is used to generate a target command in response to a processing request of a target task, and send the target command to the adapter 21 through the kernel-mode driver 5; wherein the target processor 41 is a processor among multiple processors that matches the target application 3; the target task is a GPU task initiated by the target application 3, the target command is a command corresponding to the target task, and the target command contains physical GPU resource requirement information and / or CPU resource requirement information required to process the target task.

[0103] The resource scheduling module 211 is used to allocate the target command to the target scheduling unit 2111; wherein the target scheduling unit 2111 is a scheduling unit that matches the target application 3 among multiple scheduling units.

[0104] The target scheduling unit 2111 is used to obtain the physical GPU resource available information and / or CPU resource available information of the physical GPU device 1, parse the command requirement information from the target command, and allocate the physical GPU resource available information and / or CPU resource available information according to the command requirement information.

[0105] When the physical GPU resource available information is allocated, the physical GPU device 1 is called, and the physical GPU device 1 processes the target task to obtain a processing feedback result.

[0106] The embodiment of the present application can achieve abstract management of physical GPU resources through the target processor set in the user-mode driver of the client virtual machine and the adapter set in the virtual GPU device, and support the free allocation and dynamic scaling of physical GPU resources. It can flexibly allocate resources, improve the timeliness of processing feedback results, and improve the resource utilization rate of the physical GPU device.

[0107] In one possible implementation, Fig. 9 As shown, the target application 3 includes at least one of a rendering application 31 , a video application 32 , and a parallel computing application 33 .

[0108] It should be understood that the rendering application 31 is also referred to as rendering software. Fig.10 As shown, it includes but is not limited to: OpenGL applications, Vulkan applications, Direct3D applications, etc.

[0109] like Fig. 9 As shown, the target processor 41 includes at least one of a rendering processor 411 , a video processor 412 , and a parallel computing processor 413 .

[0110] It should be understood that the video processor 412 may also be referred to as a video acceleration processor, and the parallel computing processor 413 may also be referred to as a parallel computing acceleration processor.

[0111] like Fig. 9 As shown, the target scheduling unit 2111 includes: at least one of a rendering scheduling unit 221 , a video scheduling unit 222 and a parallel computing scheduling unit 223 .

[0112] That is, the adapter 21 includes a resource scheduling module 211, and the resource scheduling module 211 includes a rendering scheduling unit 221, a video scheduling unit 222 and a parallel computing scheduling unit 223. The main purpose of the adapter 21 is to calculate and schedule the resources of the physical GPU device 1 according to the corresponding data.

[0113] The target command includes at least one of a rendering command, a video parameter, and a calculation instruction.

[0114] The embodiment of the present application provides a flexible and comprehensive virtual GPU device for the client virtual machine, and realizes the virtualization of the full functions of the virtual GPU device. The various functions of the virtual GPU device include: computing acceleration, video acceleration, 3D rendering, etc. This embodiment realizes the user-mode driver calling the target application through the provided virtual GPU device, and then can realize the calling of physical GPU devices of different specific manufacturers by converting the client virtual machine.

[0115] According to the usage of physical GPU resources, this embodiment can flexibly allocate physical GPU resources so that physical GPU resources in cloud computing can be dynamically scheduled, which can not only greatly improve the usage efficiency of physical GPU devices, but also reduce the deployment cost of physical GPU devices.

[0116] The embodiment of the present application first analyzes the functions of each module on the client virtual machine, and then analyzes the functions of each module of the virtual GPU device 2, specifically:

[0117] In a possible implementation, the target processor 41 is used to generate a target command in response to a processing request of the target task, and send the target command to the adapter 21 through the kernel mode driver 5, including at least one of the following:

[0118] The rendering processor 411 generates a rendering command in response to a processing request for a rendering task initiated by the rendering application 31 , and sends the rendering command to the adapter 21 through the kernel-mode driver 5 .

[0119] like Fig.10 As shown, the rendering processor 411 is set in the user mode driver of the client virtual machine, and there is a corresponding scheduling unit in the virtual GPU device 2 to cooperate with it to achieve the management and processing of the rendering tasks initiated by the client virtual machine.

[0120] Depend on Fig.10 It can be seen that: (1) The application of the rendering processor can be realized through the user-mode driver, kernel-mode driver and driver loader. The above-mentioned driver loader is used to select the driver entry function address corresponding to the rendering processor. (2) The user-mode driver starts the rendering manager, executes the processing flow of the target task initiated by OpenGL applications, Vulkan applications, Direct3D applications, etc., and manages the physical GPU resources. (3) The user-mode driver is uniformly issued the drawcall command and resource data (i.e. the above-mentioned physical GPU resource requirement information and / or CPU resource requirement information) by the rendering manager.

[0121] The video processor 412 generates video parameters in response to a processing request of a video task initiated by the video application 32, and sends the video parameters to the adapter through a kernel-mode driver.

[0122] It should be understood that the above-mentioned video parameters may also be called video parameter information.

[0123] Combination Fig.10 and Fig.11It can be seen that the video processor is set in the user-mode driver of the client virtual machine, and the video processor is used to: (1) receive calls to the video processor through a calling interface; (2) the video processor receives data called from the video application; (3) parse the data and process the video parameters; (4) call the rendering manager function to generate a picture buffer; (5) call the feature processing unit according to the video parameters to support multiple mainstream video formats; (6) parse data units such as macroblocks to support multiple mainstream video formats; (7) call the corresponding encoder or decoder function according to functional requirements and video types to support multiple mainstream video formats; (8) generate the corresponding encoding / decoding command stream; (9) write the corresponding video frame data to the frame buffer (i.e., the data cache); (10) optionally, start the device computing adapter, which is set in the parallel computing processor to receive data and call the parallel computing processor for fast computing.

[0124] The parallel computing processor generates computing instructions in response to a processing request for a parallel computing task initiated by a parallel computing application, and sends the computing instructions to the adapter through a kernel-mode driver.

[0125] like Fig.10 and Fig.11 As shown, the parallel computing processor is set in the user-mode driver of the client virtual machine and is mainly used to: (1) receive calls for parallel computing applications; (2) generate computing kernels; (3) call the shader in the rendering manager to generate an intermediate representation language; (4) call the rendering manager to process image generation; (5) allocate computing resources corresponding to the required data; and (6) submit computing commands and data.

[0126] Compared with the existing virtual GPU implementation scheme, the virtual GPU implementation in this embodiment is implemented through a complete virtual GPU device, connected to the wddm driver framework of the Windows system (including user-mode driver wddm umd and user-mode driver wddm umd), supports standard rendering and video encoding and decoding functions on Windows, and ensures the functional integrity of the overall operation of the client virtual machine. The above integrity is reflected in the completeness of functions and the completeness of multi-operating system applicability.

[0127] In a possible implementation, the rendering processor 411 is further used to convert the target task initiated by the rendering application 31 in the first system environment into a target executable file in the second system environment, wherein the first system environment is different from the second system environment, and the physical GPU device 1 supports the second system environment.

[0128] In the embodiment of the present application, the first system environment may refer to a Linux system environment, and the second system environment may refer to a Windows system environment.

[0129] Since the above-mentioned rendering application 31 includes but is not limited to: OpenGL application, Vulkan application, Direct3D application, etc., the Direct3D application has multiple versions, and the embodiment of the present application has a conversion function, the embodiment of the present application can enable the task processing system to support the conversion of various versions of Direct3D applications and OpenGL applications and / or Vulkan applications.

[0130] Combination Fig.11 It can be seen that the embodiments of the present application can convert the rendering tasks of Direct3D applications into rendering tasks of OpenGL applications / Vulkan applications. For example, (1) various versions of applications Direct3D_9, Direct3D_10, and Direct3D_11 can be converted into related calls of OpenGL; (2) the application Direct3D_12 can be converted into the rendering task of the Vulkan application for calling; (3) the coordinate system and clipping space of Direct3D are converted; (4) the Direct3D viewport conversion can be realized; (5) the high-level shader language HLSL is converted into the shading language GLSL in OpenGL; (6) the shading language DXIL is converted into the high-level intermediate language SPIRV in the Vulkan application.

[0131] Furthermore, the embodiment of the present application forms 3D function conversion logic through the rendering processor set in the user-mode driver and the adapter set in the virtual GPU device, thereby completing the conversion between various applications and maximizing the rendering support for various system environments, for example: supporting the running of Direct3D applications in the Linux system environment.

[0132] In one possible implementation, Fig.11 As shown, the kernel-mode driver includes a command buffer and a data buffer.

[0133] The command buffer is used to cache the target commands obtained from the target processor and schedule the pending command queue; wherein the pending command queue is composed of the cached target commands.

[0134] Data cache area, used to cache the data of the target task.

[0135] Combination Fig.10 and Fig.11 It can be seen that the kernel-mode driver is set in the user-mode driver of the client virtual machine and is used to: (1) allocate command buffers and data spaces (i.e., the above-mentioned data buffers); (2) obtain commands and data submitted by the user-mode driver; (3) submit commands and data to the virtual GPU device; (4) process frame synchronization information; and (5) schedule the pending command queue consisting of pending commands in the command buffer.

[0136] After describing the functions of each module of the client virtual machine, the embodiment of the present application analyzes each module on the virtual GPU device as follows:

[0137] according to Figure 8 and Fig. 9 It can be seen that the embodiment of the present application analyzes the function of the adapter 21 on the virtual GPU device 2 as follows:

[0138] (1) The adapter 21 is set in the virtual GPU device 2 to process various task requirements issued by the rendering processor 411, the video processor 412 and the parallel computing processor 413 in the client virtual machine, and manage such task requirements with the processing process of the physical GPU device 1.

[0139] (2) After receiving the task data sent by the rendering processor 411, the video processor 412 and the parallel computing processor 413 in the client virtual machine, the adapter 21 performs internal processing. The specific process of the internal processing is as follows: Fig.10 and Fig.11 As shown, no further description is given here.

[0140] (3) The adapter 21 can calculate and allocate GPU resources according to the received data through the resource scheduling module 211 included therein. The GPU resources include physical GPU resources and virtual GPU resources.

[0141] (4) The adapter 21 can analyze the processing capability of the physical GPU device 1, and determine the capability value of the resource scheduling module 211, such as the task queue size, the maximum number of video frames (FramesPerSecond, fps) supported, etc., based on the video memory size, the number of stream processors, the number of rasterization engines, etc. of the physical GPU device 1.

[0142] (5) The adapter 21 is used to encapsulate the rendering scheduling unit 221, the video scheduling unit 222 and the parallel computing scheduling unit 223, so that each scheduling unit processes different types of tasks.

[0143] (6) The adapter 21 can assign different types of tasks to different functional logic units (i.e., the rendering scheduling unit 221, computing scheduling unit or video scheduling unit 222 mentioned above), and the assigned scheduling unit is used to abstractly encapsulate the logic of the physical GPU device 1.

[0144] (7) Adapter 21, used to parse specific rendering commands in the rendering scheduling unit 221, and control the physical GPU device 1 to allocate resources such as texture and buffer through the functional interface provided by the driver of the physical GPU device 1, and then process the specified rendering task.

[0145] (8) The adapter 21 is also used to parse the video parameters in the video scheduling unit 222, and send a processing request corresponding to the video task to the physical GPU device 1 through the data cache module, so that the physical GPU device 1 allocates physical GPU resources such as the video buffer corresponding to the video task.

[0146] (9) The adapter 21 is also used to parse the computing instructions in the parallel computing scheduling unit 223, encapsulate the instructions corresponding to the physical GPU resource requirement information, and send the instructions to the physical GPU device 1.

[0147] (10) The adapter 21 is also used to periodically measure the task processing load in each scheduling unit and dispatch tasks to each scheduling unit according to a set strategy. For example, if the number of vertices per unit time exceeds a preset threshold, the rendering task is suspended to avoid task congestion.

[0148] (11) The adapter 21 is also used to write back the processing feedback results obtained by each scheduling unit by calling the physical GPU device 1 to process the target task in the virtual GPU device 2 interrupt task and its delayed scheduling program.

[0149] The adapter in the embodiment of the present application can adapt to the processors of various functions, cache data / commands, manage rendering and computing status, manage physical GPU resources, and allocate physical GPU resources to rendering tasks, video tasks, parallel computing tasks, etc., to achieve reasonable scheduling of physical GPU resources.

[0150] In a possible implementation, in the adapter 21, the target scheduling unit 2111 is further configured to perform at least one of the following:

[0151] The rendering scheduling unit 221 is used to parse the rendering requirement information of the rendering task from the rendering command.

[0152] The video scheduling unit 222 is used to parse the video requirement information of the video task from the video parameters.

[0153] The parallel computing scheduling unit 223 is used to parse the computing requirement information of the parallel computing task from the computing instruction.

[0154] Compared with the existing virtual GPU technology implemented by binding fixed physical GPU resources, the virtual GPU technology in this embodiment can achieve full virtualization, and it is implemented by all software. That is to say, this embodiment can achieve virtualization of rendering, parallel computing, and video acceleration processing, so that it does not rely on binding fixed physical GPU resources. Through the rendering processor, parallel computing processor, video processor set in the user state driver of the client virtual machine and the rendering scheduling unit, video scheduling unit and parallel computing scheduling unit in the adapter set in the virtual GPU device, the physical GPU device is abstractly managed (including but not limited to: GPU instruction conversion, data format conversion, resource management, etc.), and while achieving full functional support for the virtual GPU device, it can also support the free allocation and dynamic scaling of physical GPU resources.

[0155] In one possible implementation, Fig. 9 and 11 As shown, the adapter 21 further includes: a command cache module 212 and a data cache module 213 .

[0156] The command cache module is used to cache the target commands obtained from the command cache area; the data cache module is used to cache the target task data obtained from the data cache area, as well as the processing feedback results of the target task.

[0157] The command buffer module is also called a queue buffer module, and the data buffer module is also called a data buffer module.

[0158] On the basis that the virtual GPU device includes an adapter, the virtual GPU device in the embodiment of the present application can implement the following functions:

[0159] (1) The virtual GPU device is used to allocate physical GPU resources; wherein the physical GPU device corresponding to the virtual GPU device has a memory vram for managing the allocation of physical GPU resources, which can be adaptively used according to different configurations and functions.

[0160] (2) After command parsing, the virtual GPU device allocates rendering and / or computing resources according to the resource requirements of the target task corresponding to the target command. The resources include physical GPU resources and virtual GPU resources.

[0161] (3) The virtual GPU device is also used to process the command list in the command cache module. The command list is a command stream in the command cache module and is used to cache commands for rendering processing, video acceleration processing, and parallel computing processing.

[0162] (4) The virtual GPU device is also used to process frame synchronization information, which may include video acceleration processing and 3D rendering processing.

[0163] (5) Virtual GPU devices are also used to manage rendering processes, parallel computing processes, and video processing processes. Specifically, virtual GPU devices are used to manage rendering processes, such as drawcall processing status, resource data (such as texture, buffer, etc.) processing progress, rendering result writeback, etc.; virtual GPU devices are also used to manage parallel computing processes, such as kernel function computeshader status detection, resource settings, parallel computing task distribution, etc.; virtual GPU devices are also used to manage video processing processes, such as parameter preparation and cache management corresponding to formats such as h264 / h265.

[0164] (6) The virtual GPU device is also used to manage physical GPU resources. Specifically, this embodiment allocates resources such as texture and buffer from physical GPU resources according to the needs of running various functional scheduling modules.

[0165] Combine the following Fig.12 , the scheduling process is analyzed as follows:

[0166] Step 121: The virtual GPU device receives commands and data from a command buffer and a data buffer provided by a kernel-mode driver.

[0167] Step 122: Receive video parameters and frame information from the data buffer area provided by the kernel-mode driver.

[0168] Step 123: The virtual GPU device may perform command parsing through a built-in resource scheduling module.

[0169] Step 124: After the command parsing is completed, the virtual GPU device calculates resource allocation.

[0170] When computing resource allocation, the embodiment of the present application is provided with the following computing resource allocation logic: physical GPU resources are allocated according to the command requirement information corresponding to the target task, specifically:

[0171] When the command requirement information is rendering requirement information, the GPU resources required for processing the rendering requirement information are calculated, and an equal amount of physical GPU resources are provided as GPU rendering resources.

[0172] When the command requirement information is video requirement information, the GPU resources required for processing the video requirement information are calculated, and an equal amount of physical GPU resources are provided as GPU video resources.

[0173] When the command requirement information is computing requirement information, the GPU resources required for processing the computing requirement information are calculated, and an equal amount of physical GPU resources are provided as GPU computing resources.

[0174] Step 125, 1) the rendering scheduling unit in the virtual GPU device is used to coordinate and adapt the GPU rendering resources, submit rendering commands, and collect processing results; 2) the video scheduling unit in the virtual GPU device is used to coordinate and adapt the GPU video processing resources, submit rendering commands, and collect processing results; 3) the parallel computing scheduling in the virtual GPU device is used to coordinate and adapt the GPU computing resources, submit computing commands, and collect processing results.

[0175] Step 126: Write the processing feedback result to the data cache module.

[0176] The above steps 124 to 125 together complete the abstraction of physical GPU resources to achieve dynamic management of physical GPU resources.

[0177] The embodiments of the present application have the following advantages: (1) The embodiments can improve the compatibility of the operating system software running in the client virtual machine and can support various 3D applications, such as: 3D modeling, animation production, simulation, VR / AR, video playback, etc.; (2) The embodiments can improve the graphics operation performance of the operating system in the client virtual machine. Under the same computing resources, the smoothness of ordinary graphics operations is improved by more than 60%; (3) The embodiments can reduce the construction cost of the existing virtual GPU technology solution and achieve a cost saving of more than 300%.

[0178] In addition, the functions included in the embodiments of the present application are not limited to: virtualization of ordinary GPU rendering, video acceleration, parallel computing acceleration and other functions, and it can expand more other computing acceleration support. For example, this embodiment can expand the virtualization support for AI computing acceleration dedicated chips.

[0179] Embodiment 2:

[0180] Based on the above embodiments, the technical solution of the present application is described in more detail below in conjunction with specific embodiments.

[0181] Fig.13 A schematic diagram of the structure of a virtual GPU architecture provided in an embodiment of the present application. The virtual GPU architecture can be understood as the architecture of another task processing system.

[0182] The virtual GPU architecture is provided with a virtual GPU device and a physical GPU device on the host. Different client virtual machines may include Linux virtual machines and Windows virtual machines. A Linux application is provided on the Linux virtual machine. Each software initiates a corresponding task to the user-state driver mesa2 in the runtime OpenGL runtime, OpenCL runtime and Libva runtime. The user-state driver mesa2 is deployed with processors corresponding to each function (including a rendering processor, a video processor and a parallel computing processor), and sends the corresponding commands and data to the virtual GPU device via the kernel-state driver drm2.

[0183] Among them, Windows applications are set up on the Windows virtual machine. Each software initiates the corresponding task to the user-state driver wddm umd in the runtime OpenGL runtime, OpenCL runtime and Dxva runtime. The user-state driver wddm umd is deployed with processors corresponding to each function (including rendering processor, video processor and parallel computing processor), and sends the corresponding commands and data to the virtual GPU device through the kernel-state driver wddm kmd.

[0184] The virtual GPU device is deployed with an adapter, and the adapter is deployed with scheduling units corresponding to various functions (including rendering units, video units, and parallel computing units). The host is also deployed with user-mode driver mesa1 and kernel-mode driver drm1. Through the sequential action of the adapter, user-mode driver mesa1, and kernel-mode driver drm1, the resource call of the physical GPU device is realized.

[0185] In the implementation process of on-demand calling of physical GPU resources, the embodiment of the present application provides the following solutions:

[0186] (1) This embodiment is based on the driver (for example, user-mode driver umd, kernel-mode driver drm, etc.) and the virtual GPU device to work together to implement the call of the virtual GPU device to the physical GPU resources. The above-mentioned collaboration process is shown in the above-mentioned embodiment. Fig.11 , I will not go into details here.

[0187] (2) Rendering adaptation, video acceleration adaptation, and parallel computing adaptation are implemented through the scheduling units provided by the adapter, so as to implement rendering, video acceleration, parallel computing acceleration and other functions in the virtual machine, thereby realizing various logical functions of the virtual GPU device.

[0188] (3) The client virtual machine includes a specific functional unit driven by a virtual GPU device (see Fig.12 Modules in the .

[0189] (4) Virtual GPU devices provide virtual GPU function support for audio and video encoding / decoding and parallel computing acceleration.

[0190] (5) This embodiment implements support for 3D rendering applications such as OpenGL applications, Direct3D applications, and Vulkan applications in the system environment of the client virtual machine. Functional docking can be completed by a function converter, and the input of the function converter is various 3D applications to be converted.

[0191] (6) This embodiment implements virtualization of the virtual GPU device on the host, and can complete the abstract management of the hardware resources of the physical GPU device.

[0192] (7) This embodiment can implement calls / conversions between applications through the renderer processor, video processor, parallel computing processor, etc. deployed in the user-mode driver of the Windows virtual machine, so that the virtual GPU device supports both Windows virtual machines and Linux virtual machines.

[0193] (8) This embodiment supports on-demand use of physical GPU resources, thereby improving the utilization rate of physical GPU resources.

[0194] In summary, the virtual GPU architecture can provide a general virtual GPU solution, and it can realize the various functions of the virtual GPU device based on virtualization technology. The advantage is that it can be separated from the binding of the physical GPU device and can run independently, supporting the flexible allocation of physical GPU resources.

[0195] Embodiment 3:

[0196] Fig.14 A flowchart of a task processing method provided in an embodiment of the present application is provided. Figure 8 The task processing system shown in the figure includes: a virtual GPU device located on a host machine, and a target application and a virtual GPU driver deployed on a client virtual machine, wherein the virtual GPU driver virtual program includes a user-mode driver and a kernel-mode driver; wherein a plurality of processors are arranged in the user-mode driver; an adapter is arranged in the virtual GPU device, the adapter includes a resource scheduling module, and the resource scheduling module includes a plurality of scheduling units.

[0197] like Fig.14 As shown, the above task processing method includes the following steps:

[0198] S1. A target processor set in a user-mode driver of a client virtual machine generates a target command in response to a processing request of a target task, and sends the target command to an adapter set in a virtual GPU device through a kernel-mode driver set on the client virtual machine. The target processor is a processor that matches a target application among multiple processors; the target task is a GPU task initiated by a target application; the target command is a command corresponding to the target task, and the target command contains physical GPU resource requirement information and / or CPU resource requirement information required to process the target task.

[0199] S2. Allocate the target command to the target scheduling unit through the resource scheduling module, wherein the target scheduling unit is a scheduling unit that matches the target application among the multiple scheduling units.

[0200] S3. Parsing command requirement information from the target command through the target scheduling unit, and allocating physical GPU resource availability information and / or CPU resource availability information according to the command requirement information.

[0201] S4. When the physical GPU resource available information and / or the CPU resource available information is allocated, the target task is processed by the physical GPU device and / or the CPU device to obtain a processing feedback result.

[0202] The physical GPU resource available information is used to reflect the GPU available resources, and the CPU resource available information is used to reflect the CPU available resources. The target task is a GPU task. For the target task, this embodiment has a preset resource usage strategy. The main design idea of ​​the preset resource usage strategy is to give priority to the use of physical GPU devices. Specifically:

[0203] If the available GPU resources are higher than the resources required to process the target task, the physical GPU device is used for processing.

[0204] If the available GPU resources are higher than the preset threshold but lower than the resources required to process the task, the physical GPU device and the CPU device are used together for processing.

[0205] If the GPU resources available are lower than the preset threshold, the CPU device is used for processing.

[0206] Optionally, the target application includes: at least one of a rendering application, a video application and a parallel computing application; the target processor includes: at least one of a rendering processor, a video processor and a parallel computing processor; the target scheduling unit includes: at least one of a rendering scheduling unit, a video scheduling unit and a parallel computing scheduling unit; the target command includes: at least one of a rendering command, a video parameter and a computing instruction.

[0207] Optionally, S1, generating a target command by a target processor set in a user-mode driver of the client virtual machine in response to a processing request of a target task, and sending the target command to an adapter set in a virtual GPU device by a kernel-mode driver set on the client virtual machine, includes at least one of the following:

[0208] The rendering processor generates a rendering command in response to a processing request for a rendering task initiated by a rendering application, and sends the rendering command to the adapter through a kernel-mode driver.

[0209] The video processor generates video parameters in response to a processing request of a video task initiated by a video application, and sends the video parameters to the adapter through a kernel-mode driver.

[0210] The parallel computing processor generates computing instructions in response to the processing request of the parallel computing task initiated by the parallel computing application, and sends the computing instructions to the adapter through the kernel mode driver.

[0211] Optionally, S3, parsing command requirement information from the target command by the target scheduling unit, including at least one of the following:

[0212] The rendering requirement information of the rendering task is parsed from the rendering command through the rendering scheduling unit.

[0213] The video requirement information of the video task is parsed from the video parameters through the video scheduling unit.

[0214] The computing requirement information of the parallel computing task is parsed from the computing instructions through the parallel computing scheduling unit.

[0215] Optionally, the method further comprises:

[0216] S5. Converting a target task initiated by the rendering application in the first system environment into a target executable file in the second system environment through a rendering processor; wherein the first system environment is different from the second system environment, and the physical GPU device supports the second system environment.

[0217] Optionally, the kernel-mode driver includes a command buffer and a data buffer; the method further includes:

[0218] S6. Cache the target command obtained from the target processor through the command buffer area, and schedule the pending command queue; wherein the pending command queue is composed of the cached target commands.

[0219] S7. Cache the data of the target task through the data buffer area.

[0220] Optionally, the adapter further includes: a command cache module and a data cache module; the method further includes:

[0221] S8. Cache the target command obtained from the command buffer area through the command buffer module.

[0222] S9. Cache the target task data obtained from the data cache area and the processing feedback result of the target task through the data cache module.

[0223] Optionally, the virtual GPU device is further used to perform any of the following:

[0224] Allocate physical GPU resources; wherein the physical GPU device corresponding to the virtual GPU device has a memory vram for managing the allocation of physical GPU resources;

[0225] After command parsing, rendering and / or computing resources are allocated according to the resource requirements of the target task corresponding to the target command. The resources include physical GPU resources and virtual GPU resources.

[0226] Process the command list in the command cache module. The command list is a command stream in the command cache module, which is used to cache rendering commands, video parameters, and calculation instructions;

[0227] Processing frame synchronization information, where the frame synchronization information is used to implement video acceleration processing and / or 3D rendering processing;

[0228] Manage the state of target processing operations, including rendering, parallel computing, and video processing;

[0229] Manage physical GPU resources.

[0230] The implementation principle and technical effect of the task processing method provided in this embodiment are similar to those of the above-mentioned system embodiment, and will not be described in detail here.

[0231] It should be noted that the user information and data involved in this application (including but not limited to data used for analysis, stored data, displayed data, etc.) are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0232] That is to say, in the technical solution of this application, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the relevant laws and regulations and do not violate public order and good morals.

[0233] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0234] Fig.15A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device includes a receiver 60, a transmitter 61, at least one processor 62 and a memory 63. The electronic device composed of the above components can be used to implement the above-mentioned several specific embodiments of the present application, which will not be described in detail here.

[0235] The embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, each step of the method in the above embodiment is implemented.

[0236] An embodiment of the present application also provides a computer program product, including a computer program, which implements each step of the method in the above embodiment when executed by a processor.

[0237] Various embodiments of the systems and techniques described above in the present application can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs, which can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor, which can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0238] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or electronic device.

[0239] In the context of the present application, a computer-readable storage medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may be a machine-readable signal medium or a machine-readable storage medium. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a computer-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0240] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0241] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data electronic device), or a computing system that includes middleware components (e.g., an application electronic device), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0242] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps disclosed in this application can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in this application can be achieved, and this document does not limit this.

[0243] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the principles of this application should be included in the protection scope of this application.

Claims

1. A task processing method, characterized in that: include: The target processor set in the user-mode driver of the client virtual machine responds to the processing request of the target task, generates a target command, and sends the target command to the adapter set in the virtual GPU device through the kernel-mode driver set on the client virtual machine; wherein the target processor is a processor matching the target application among multiple processors; the target task is a GPU task initiated by the target application, and the target command is a command corresponding to the target task; Allocating the target command to a target scheduling unit through a resource scheduling module; wherein the target scheduling unit is a scheduling unit that matches the target application among multiple scheduling units; Parsing command requirement information from the target command through the target scheduling unit, and allocating physical GPU resource availability information and / or CPU resource availability information according to the command requirement information; When the physical GPU resource available information and / or the CPU resource available information are allocated, the target task is processed by the physical GPU device and / or the CPU device to obtain a processing feedback result.

2. The method according to claim 1, characterized in that The target application includes: at least one of a rendering application, a video application, and a parallel computing application; The target processor includes: at least one of a rendering processor, a video processor, and a parallel computing processor; The target scheduling unit includes: at least one of a rendering scheduling unit, a video scheduling unit, and a parallel computing scheduling unit; The target command includes at least one of a rendering command, a video parameter and a calculation instruction.

3. The method according to claim 2, characterized in that The generating a target command by the target processor in response to the processing request of the target task, and sending the target command to the adapter by the kernel mode driver, includes at least one of the following: Generate the rendering command by the rendering processor in response to the processing request of the rendering task initiated by the rendering application, and send the rendering command to the adapter by the kernel state driver; Generate the video parameters by the video processor in response to the processing request of the video task initiated by the video application, and send the video parameters to the adapter by the kernel state driver; The parallel computing processor generates the computing instruction in response to the processing request of the parallel computing task initiated by the parallel computing application, and sends the computing instruction to the adapter through the kernel mode driver.

4. The method according to claim 2, characterized in that: The step of parsing command requirement information from the target command by the target scheduling unit includes at least one of the following: Parsing rendering requirement information of the rendering task from the rendering command through the rendering scheduling unit; parsing video requirement information of the video task from the video parameters by the video scheduling unit; The parallel computing scheduling unit parses the computing instruction to obtain computing requirement information of the parallel computing task.

5. The method according to claim 2, characterized in that: The method further comprises: The rendering processor converts a target task initiated by the rendering application in the first system environment into a target executable file in the second system environment; wherein the first system environment is different from the second system environment, and the physical GPU device supports the second system environment.

6. The method according to claim 1, characterized in that The kernel-mode driver includes a command buffer and a data buffer; The method further comprises: Cache the target command obtained from the target processor through the command buffer area, and schedule a queue of commands to be processed; wherein the queue of commands to be processed is composed of the cached target commands; The data of the target task is cached through the data cache area.

7. The method according to claim 1, characterized in that The adapter also includes: a command cache module and a data cache module; The method further comprises: caching the target command obtained from the command buffer area by the command buffer module; The data of the target task acquired from the data cache area and the processing feedback result of the target task are cached by the data cache module.

8. The method according to any one of claims 1 to 7, characterized in that: The virtual GPU device is further configured to perform any of the following: Allocating physical GPU resources; wherein the physical GPU device corresponding to the virtual GPU device has a memory vram for managing the allocation of physical GPU resources; After command parsing, rendering and / or computing resources are allocated according to the resource requirements of the target task corresponding to the target command, wherein the resources include physical GPU resources and virtual GPU resources; Processing a command list in the command cache module, wherein the command list is a command stream in the command cache module and is used to cache rendering commands, video parameters, and calculation instructions; Processing frame synchronization information, wherein the frame synchronization information is used to implement video acceleration processing and / or 3D rendering processing; Managing the state of target processing operations including rendering, parallel computing, and video processing; Managing the physical GPU resources.

9. A task processing system, characterized in that: The task processing system includes: a virtual GPU device located on a host machine, and a target application and a virtual GPU driver deployed on a client virtual machine, wherein the virtual GPU driver virtual program includes a user state driver and a kernel state driver; wherein the user state driver is provided with a plurality of processors; an adapter is provided in the virtual GPU device, wherein the adapter includes a resource scheduling module, and the resource scheduling module includes a plurality of scheduling units; a target processor, for generating a target command in response to a processing request of a target task, and sending the target command to the adapter through the kernel-mode driver; wherein the target processor is a processor among the multiple processors that matches the target application; the target task is a GPU task initiated by the target application, and the target command is a command corresponding to the target task; The resource scheduling module is used to allocate the target command to a target scheduling unit; wherein the target scheduling unit is a scheduling unit that matches the target application among the multiple scheduling units; The target scheduling unit is used to parse command requirement information from the target command, and allocate physical GPU resource availability information and / or CPU resource availability information according to the command requirement information; When the physical GPU resource available information and / or the CPU resource available information are allocated, the target task is processed by the physical GPU device and / or the CPU device to obtain a processing feedback result.

10. An electronic device, characterized in that: include: at least one processor and memory; The memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the task processing method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the task processing method according to any one of claims 1 to 8.

Images