A window rendering acceleration method, device and computing device

Abstract

The application discloses a window rendering acceleration method and device and a computing device, and relates to the technical field of GPU rendering. The window rendering acceleration method comprises the following steps: in response to the starting of a graphical user interface application program, judging a window rendering mode; when the result of the judgment is that the graphic processor is used for rendering, loading a picture into a display memory and parsing the picture into a texture by calling an interactive interface of a kernel; superimposing the texture on a window picture, rendering the window picture superimposed with the texture into the display memory to obtain display memory picture data; and submitting the display memory picture data to a window manager for rendering. The method can effectively reduce the CPU resource occupancy and improve the GPU rendering efficiency.

Description

Technical Field

[0001] This invention relates to the field of GPU rendering technology, and in particular to a window rendering acceleration method, apparatus and computing device. Background Technology

[0002] In UI (User Interface) frameworks provided by Qt (a cross-platform C++ graphical user interface application development framework) or GTK (GIMP Toolkit, another cross-platform C++ graphical user interface application development framework), the display of the interface is typically rendered using the Central Processing Unit (CPU). Therefore, the image loading methods provided by the aforementioned UIs directly load images into memory. Thus, in such UI frameworks, when rendering the interface using the Graphics Processing Unit (GPU), the image data in the interface needs to be copied to video memory and then converted into data that the GPU can recognize before the rendering process can be completed.

[0003] like Figure 1 As shown, after the Graphical User Interface (GUI) program starts, it executes the following steps: 1. Call the memory allocation (malloc) function to allocate memory for the image and load the image into memory; 2. Determine the rendering mode; 3. If the rendering mode is CPU rendering, render the window image into memory, and then submit the image data to the window manager to complete the rendering; 4. If the rendering mode is GPU rendering, call the texture generation function (glTexlmagte2D) to generate a texture based on the image in memory, then copy the image in memory to video memory, render the window image into video memory based on the texture image stored in video memory, and then submit the image data to the window manager to complete the rendering.

[0004] The purpose of using GPU rendering is to reduce CPU resource consumption, thereby improving system performance. However, as we know from the rendering process described above, image data loaded into memory needs to be copied to video memory before it can be used by the GPU. This adds an extra memory copying process when using GPU rendering. This memory copying consumes CPU resources, so the CPU usage will not only not decrease significantly, but will actually increase when there are many image elements on the screen.

[0005] In summary, the existing GPU rendering technologies mentioned above have the problem of not being able to effectively reduce CPU resource consumption. Summary of the Invention

[0006] To this end, the present application provides a window rendering acceleration method, device and computing device to try to solve or at least alleviate at least one of the above problems.

[0007] According to an aspect of the present application, a window rendering acceleration method is provided, comprising: in response to a graphical user interface application starting, judging a window rendering mode; when the result of the judgment is to use a graphic processor to render, loading a picture into a video memory and parsing it into a texture by calling an interactive interface of a kernel; superimposing the texture on a window picture and rendering the window picture superimposed with the texture into the video memory to obtain video memory picture data; and submitting the video memory picture data to a window manager for rendering.

[0008] Optionally, in the window rendering acceleration method according to the present application, the loading of the picture into the video memory and the parsing of it into the texture comprise: creating a cache of the video memory through a direct rendering manager; decoding picture data into the cache of the video memory; converting the cache of the decoded picture data into a memory handle of direct storage access; and generating a texture of the picture according to the memory handle of direct storage access.

[0009] Optionally, in the window rendering acceleration method according to the present application, the method decodes the picture data into the cache of the video memory through a hard decoding mode.

[0010] Optionally, in the window rendering acceleration method according to the present application, the generation of the texture of the picture according to the memory handle of direct storage access comprises: calling a texture generation function and taking the memory handle of direct storage access as an input of the texture generation function; and taking an output of the texture generation function as an identity recognition code of the texture.

[0011] Optionally, in the window rendering acceleration method according to the present application, the judgment of the window rendering mode comprises: judging whether to use the graphic processor to render according to an environment variable of the graphical user interface application.

[0012] Optionally, in the window rendering acceleration method according to the present application, the method further comprises: when the result of the judgment is to use a central processing unit to render, loading the picture into a memory; rendering a window picture into the memory to obtain memory picture data; and submitting the memory picture data to the window manager.

[0013] According to another aspect of the present application, there is also provided a window rendering acceleration method device, comprising: a judging unit adapted to judge a window rendering mode in response to a graphical user interface application starting; a first loading unit adapted to load a picture into a video memory and parse it into a texture by calling an interactive interface of a kernel when the result of the judgment is to use a graphic processor to render; a first rendering unit adapted to superimpose the texture on a window picture and render the window picture superimposed with the texture into the video memory to obtain video memory picture data; and a first data submitting unit adapted to submit the video memory picture data to a window manager for rendering.

[0014] Optionally, in the window rendering acceleration device according to the present application, the loading of the picture into the video memory and the parsing of the picture into the texture comprise: creating a cache of the video memory through a direct rendering manager; decoding picture data into the cache of the video memory; converting the cache of the decoded picture data into a memory handle of direct storage access; and generating a texture of the picture according to the memory handle of the direct storage access.

[0015] According to another aspect of the present application, there is also provided a computing device, comprising: at least one processor and a memory storing program instructions; when the program instructions are read and executed by the processor, the computing device is caused to perform the window rendering acceleration method as above.

[0016] According to still another aspect of the present application, there is also provided a readable storage medium storing program instructions, when the program instructions are read and executed by a computing device, the computing device is caused to perform the window rendering acceleration method as above.

[0017] The window rendering acceleration method of the present application judges whether to use GPU rendering or CPU rendering before rendering, and if GPU rendering is used, an interactive interface of a kernel is called to directly load a picture into a video memory for use by GPU rendering.

[0018] The window rendering acceleration method, device and computing device according to the present application can achieve at least one of the following beneficial effects: the time for copying a picture from a memory to a video memory in a GPU rendering process is saved, not only the utilization rate of the GPU is improved, but also the CPU resource occupancy rate is effectively reduced, and the efficiency of GPU rendering is improved. BRIEF DESCRIPTION OF DRAWINGS

[0019] To the accomplishment of the foregoing and related ends, certain illustrative aspects are described herein in connection with the following description and the annexed drawings. These aspects are indicative, however, of but a few of the various ways in which the principles disclosed herein can be employed and is intended to include all such aspects and their equivalents. Other advantages and novel features will become apparent from the following detailed description when considered in conjunction with the drawings. The same reference numerals are generally employed throughout the drawings to reference like elements and features.

[0020] Figure 1 A flow chart of a graphical user interface program according to the background section of the present application is shown;

[0021] Figure 2 A schematic diagram of a computing device 200 according to an embodiment of the present application is shown;

[0022] Figure 3 A flow chart of a window rendering acceleration method 300 according to an embodiment of the present application is shown;

[0023] Figure 4 A schematic diagram of a window rendering acceleration method 300 according to an embodiment of the present application is shown;

[0024] Figure 5 A schematic diagram of a window rendering acceleration apparatus 500 according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0025] Exemplary embodiments of the present disclosure will be described hereinafter with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the present disclosure to those skilled in the art.

[0026] In view of the problem that the existing GPU rendering technology cannot effectively reduce CPU resource occupation, the present application provides a window rendering acceleration method, which can effectively reduce CPU resource occupation caused by picture data copying when using GPU rendering.

[0027] Figure 2 A schematic diagram of a computing device 200 according to an embodiment of the present application is shown. It should be noted that, Figure 2 The computing device 200 shown is only an example, and in practice, the computing device used to implement the window rendering acceleration method of the present application can be any model device, and its hardware configuration can be the same as that of the computing device 200 shown, Figure 2 or different from that of the computing device 200 shown. In practice, the computing device used to implement the window rendering acceleration method of the present application can add or delete hardware components of the computing device 200 shown, and the present application does not limit the specific hardware configuration of the computing device. Figure 2 Figure 2 As shown in FIG. 2, the computing device 200 includes a CPU 210, a GPU 220, a memory 230, a display 240, and an input device 250.

[0028] As shown in FIG. 2, the computing device 200 includes a CPU 210, a GPU 220, a memory 230, a display 240, and an input device 250. Figure 2 ​As shown, the computing device 200 typically includes a central processor 210, a graphics processor 220, a hard disk 230, a memory 240, and a display device interface 250.

[0029] The hard disk 230 stores program instructions for implementing the window rendering acceleration method of the embodiments of the present application, which are read and executed by the central processor 210. When rendering is performed by the central processor 210, pictures are loaded into the memory 240; when rendering is performed by the graphics processor 220, pictures are directly loaded into the video memory 250. The picture data rendered by the central processor 210 and the picture data rendered by the graphics processor 220 are finally sent to a display device through the display device interface 250 for display.

[0030] The computing device 200 can be implemented as a server, such as a file server, a database server, an application server, and a WEB server, etc., or as a personal computer including desktop computers and notebook computers. Of course, the computing device 200 can also be implemented as part of a small-size portable (or mobile) electronic device. In the embodiments according to the present application, the computing device 200 is configured to perform the window rendering acceleration method 300 according to the present application.

[0031] The window rendering acceleration method 300 according to an embodiment of the present application includes: in response to the start of a graphical user interface application, determining a window rendering mode; when the determination result is to use a graphics processor for rendering, loading pictures into a video memory and parsing them into textures by calling an interactive interface of a kernel; superimposing the textures on a window picture and rendering the window picture superimposed with the textures into the video memory to obtain video memory picture data; and submitting the video memory picture data to a window manager for rendering.

[0032] Figure 3 A flowchart of the window rendering acceleration method 300 according to an embodiment of the present application is shown. The method 300 is performed in a computing device (such as the aforementioned computing device 200). As shown, Figure 3 The method 300 starts at 310.

[0033] In 310, in response to the start of a graphical user interface application, a window rendering mode is determined.

[0034] The window of the GUI application needs to be rendered to be presented to a user. Some interfaces of the GUI application can be rendered using only a CPU, but for some special cases, such as 3D pictures or video playing, GPU rendering is needed. Taking a media player as an example, the playing picture needs to be rendered using GPU, but other contents of the player interface, such as controls, can be rendered using CPU. Therefore, essentially, it is determined whether to use GPU rendering according to the contents to be rendered.

[0035] In one embodiment, after the GUI application is started, it is determined whether to use GPU rendering according to whether the GPU driver is installed in the system, for example, it is checked whether the system supports the OpenGL driver, if the OpenGL driver is supported, GPU rendering is not used, otherwise, CPU rendering is used.

[0036] In another embodiment, part of the GUI application is embedded with an environment variable for starting GPU rendering in the program bottom, and it can be determined according to the environment variable whether the application interface uses GPU rendering before the application is started.

[0037] In addition to the above two determinations, other ways can also be used to determine whether to use GPU rendering, and the specific way of determination is not limited in the application.

[0038] Next, in 320, when the determination result is to use the graphics processor for rendering, the picture is loaded into the video memory and parsed into a texture by calling the interactive interface of the kernel.

[0039] Loading the picture into the video memory requires calling the interactive interface (ioctl) of the kernel, and ioctl belongs to the interactive interface. Through ioctl, the picture can be loaded into the video memory using the GPU driver.

[0040] In one embodiment, as shown in Figure 4 The ioctl interface loads the picture into the video memory and parses it into a texture in the following way: first, a buffer of the video memory is created through the Direct Rendering Manager (DRM), and it should be noted that the buffer here refers to a buffer of a picture; then the picture data is decoded into the created buffer, and the decoding method can be in the existing way, and the preferred method is hard decoding; next, the decoded picture data in the buffer is converted into a direct memory access (dmafd) handle through the interface of the DRM; finally, the eglCreateImageKHR interface is called, and the dmafd and eglDisplay parameters are input into the eglCreateImageKHR interface, and the eglCreateImageKHR interface makes the buffer of the picture into a texture and returns the ID of the texture.

[0041] Next, in 330, the texture is superimposed on the window picture, and the window picture superimposed with the texture is rendered into the video memory to obtain the video memory picture data. The window picture here refers to the picture finally presented to the user. Taking a media player as an example, the window picture not only contains the content played by the media player, but also includes the background, controls and other content. These contents are synthesized together to form the window picture.

[0042] First, the texture generation function glTexlmagte2D is called to generate the texture of the above image. Then, the window image to be rendered is rendered onto the above texture to obtain the video memory image data.

[0043] Next, in step 340, the video memory data is submitted to the window manager for rendering. The window manager is responsible for displaying the video memory data in the application window.

[0044] Method 300 also includes a CPU rendering process. If the result of 310 indicates that CPU rendering is to be used, the process jumps to the CPU rendering flow, such as... Figure 4 As shown.

[0045] In one embodiment, the CPU rendering process includes: allocating memory using the memory allocation function malloc and loading the image into memory; rendering all elements in the window into a memory buffer using the CPU based on the image to obtain memory image data; and submitting the memory image data to the window manager for rendering.

[0046] Method 300 determines whether to use CPU or GPU rendering before rendering. If GPU rendering is used, it directly calls the driver-related ioctl interface to load the image data into video memory, preparing for GPU rendering. Since images are not loaded into memory before method 320, the operation of copying image data from memory to video memory is omitted when the GPU creates textures. This not only improves GPU utilization and reduces CPU resource usage but also saves time copying image data, improving window rendering efficiency. The more components in the interface that contain image elements, the more pronounced this effect becomes. In terms of specific implementation, the existing libjpeg image decoding library mainly creates a decoding buffer by creating shared memory, decodes the image data into the created memory buffer, and the GPU uses glTexImage2D to make the memory buffer into a texture, and then composites the window image. During the process of accessing the memory buffer, the GPU rendering pipeline needs to copy the memory buffer to temporary video memory, and then the GPU accesses the data from the temporary video memory. However, this embodiment proposes a new ioctl. This ioctl uses the GPU to load the image data into the video memory buffer, processes the video memory buffer through DRM to obtain the dmafd, and transmits it to the GPU rendering engine in the GUI framework. During the compositing process, the GPU rendering pipeline can access the data in the video memory buffer and can directly access the data in the image decoding video memory without copying from memory to video memory.

[0047] Embodiments of the present application also provide a window rendering acceleration apparatus 500 capable of performing the processing of the window rendering acceleration method 300 as described above. In the following, the window rendering acceleration apparatus 500 is described in conjunction with Figure 5

[0048] As shown in Figure 5 , the window rendering acceleration apparatus 500 comprises a judging unit 510, a first loading unit 520, a first rendering unit 530 and a first data submitting unit 540.

[0049] The judging unit 510 is adapted to judge the window rendering mode in response to the start of the graphical user interface application.

[0050] The first loading unit 520 is adapted to load the picture into the video memory and parse it into a texture by calling the interactive interface of the kernel when the judgment result is to use the graphic processor for rendering.

[0051] The first rendering unit 530 is adapted to superimpose the texture on the window picture and render the window picture with the superimposed texture into the video memory to obtain the video memory picture data.

[0052] The first data submitting unit 540 is adapted to submit the video memory picture data to the window manager for rendering.

[0053] In one embodiment, the loading of the picture into the video memory and the parsing of it into a texture comprises: creating a cache of the video memory through a direct rendering manager; decoding the picture data into the cache of the video memory; converting the cache of the decoded picture data into a memory handle of direct storage access; and generating the texture of the picture according to the memory handle of direct storage access.

[0054] In one embodiment, the first loading unit 520 decodes the picture data into the cache of the video memory through a hardware decoding mode.

[0055] In one embodiment, the generation of the texture of the picture according to the memory handle of direct storage access comprises: calling a texture generation function and inputting the memory handle of direct storage access as an input of the texture generation function; and inputting the output of the texture generation function as an identity code of the texture.

[0056] In one embodiment, the judging unit 510 judges whether to use the graphic processor for rendering according to an environment variable of the graphical user interface application.

[0057] In one embodiment, the apparatus 500 further comprises a second loading unit, a second rendering unit and a second data submitting unit. The second loading unit is adapted to load the picture into the memory. The second rendering unit is adapted to render the window picture into the memory to obtain the memory picture data. The second data submitting unit is adapted to submit the memory picture data to the window manager for rendering.​

[0058] The various techniques described herein can be implemented in connection with hardware or software or, where appropriate, with a combination of both. Thus, the methods and apparatus of the application, or certain aspects or portions thereof, can take the form of program code (i.e., instructions) embodied in tangible media, such as removable hard disks, USB flash drives, optical tapes, CD-ROMs, or any other machine-readable storage medium, wherein, when the program code is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the subject application.

[0059] Where the program code is executed on a programmable computer, the computing device generally includes a processor, a storage medium readable by the processor (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. The storage medium is configured to store program code which implements the window rendering acceleration method of the application. The program code is executed by the processor to implement the methods of the application.

[0060] In the context of this document, a "storage medium" can be any physical medium that stores data and / or instructions for use by an instruction execution system. The storage medium can have stored thereon such computer-readable instructions.

[0061] In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the application can be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been described in detail in order to avoid obscuring the understanding of this description.

[0062] In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the application can be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been described in detail in order to avoid obscuring the understanding of this description.

[0063] Similarly, it is to be understood that the embodiments of the present application can be alternately grouped into individual embodiments, figures, or descriptions thereof, in order to streamline the disclosure and aid in the understanding of one or more of the individual inventive aspects. Br the disclosure is not to be interpreted as reflecting an intention that the claimed application requires more features than are explicitly recited in each claim. Those skilled in the art will understand that the modules, or units, or components of the devices in the examples disclosed herein can be arranged in a device as described in the examples, or alternatively can be located in one or more devices different from the devices in the examples. The modules in the foregoing examples can be combined into a module or further divided into multiple sub-modules.

[0064] Those skilled in the art will understand that the modules in the devices in the examples can be adaptively changed and disposed in one or more devices different from the examples. The modules or units or components in the examples can be combined into a module or unit or component, and further can be divided into multiple sub-modules or sub-units or sub-components. All combinations of all features disclosed in the specification (including the accompanying claims, abstract and drawings), and any method or device so disclosed, can be taken in any combination, excepting that at least some of such features and / or steps are mutually exclusive from each other. Each feature disclosed in the specification (including the accompanying claims, abstract and drawings) can be replaced by alternative features supporting the same, equivalent, or similar purpose, unless otherwise expressly stated.

[0065] Furthermore, those skilled in the art will recognize that the exemplary embodiments described herein can be modified in arrangement and detail without departing from the intended application scope. Therefore, many modifications and variations of the present application are possible and are intended to be covered by the foregoing disclosure. Examples of the described embodiments are meant as examples and are not intended to be limiting. In addition, it is contemplated that some of the embodiments described herein can include certain features of the described embodiments that are not included in other embodiments described herein, dependent on the embodiment in question. The combination of features of different embodiments means that within the scope of the application and form different embodiments. Furthermore, some of the described embodiments are described herein as combinations of methods or method elements implemented by a processor of a computer system or by other means for performing the described functions. Therefore, a processor having the necessary instructions for performing the described methods or method elements forms a means for performing the methods or method elements. Furthermore, elements of the described embodiments are examples of means for performing the functions performed by those elements for the purpose of carrying out the application.

[0066] As used herein, unless otherwise indicated, the use of the ordinal adjectives "first", "second", "third", etc., are to be construed to each refer to a different one of the ordinal number of items. For example, "first", "second", "third", etc., items referring to different ones of the items within the list "first", "second", "third", etc. For example, "first", "second", "third", etc., items referring to different ones of the items within the list "first", "second", "third", etc.

[0067] While the application has been described in terms of several embodiments, it will be apparent to those of ordinary skill in the art that many modifications, additions, substitutions, and the like can be made to the applications set forth herein without departing from the scope of the application as set forth in the claims. Furthermore, it should be noted that the language used in the specification has been principally selected for readability and instructional purposes and can not have been selected to expressly convey the scope of the subject matter of the application.

Claims

1. A method for window rendering acceleration, comprising: judging a window rendering mode in response to a graphical user interface application starting; loading a picture into a video memory and resolving the picture into a texture by calling an interactive interface of a kernel when a result of the judging is that a graphic processor is used for rendering, wherein the loading the picture into the video memory and resolving the picture into the texture comprises: creating a cache of the video memory by a direct rendering manager, decoding picture data into the cache of the video memory, converting the cache of the decoded picture data into a memory handle of direct storage access, and generating a texture of the picture according to the memory handle of the direct storage access; superimposing the texture on a window picture and rendering the window picture with the texture superimposed into the video memory to obtain video memory picture data; and submitting the video memory picture data to a window manager for rendering.

2. The method of claim 1, wherein, The method decodes picture data into the cache of the video memory by a hard decoding mode.

3. The method of claim 1, wherein, The generating the texture of the picture according to the memory handle of the direct storage access comprises: calling a texture generation function and inputting the memory handle of the direct storage access as an input of the texture generation function; and inputting an output of the texture generation function as an identity recognition code of the texture.

4. The method of claim 1, wherein, The judging the window rendering mode comprises: judging whether the graphic processor is used for rendering according to an environment variable of the graphical user interface application. 5.The method of claim 1, further comprising: loading the picture into a memory when a result of the judging is that a central processing unit is used for rendering; rendering the window picture into the memory to obtain memory picture data; and submitting the memory picture data to the window manager for rendering. 6.An apparatus for window rendering acceleration, comprising: a judging unit adapted to judge a window rendering mode in response to a graphical user interface application starting; a first loading unit adapted to load a picture into a video memory and resolve the picture into a texture by calling an interactive interface of a kernel when a result of the judging is that a graphic processor is used for rendering, wherein the loading the picture into the video memory and resolving the picture into the texture comprises: creating a cache of the video memory by a direct rendering manager, decoding picture data into the cache of the video memory, converting the cache of the decoded picture data into a memory handle of direct storage access, and generating a texture of the picture according to the memory handle of the direct storage access; a first rendering unit adapted to superimpose the texture on a window picture and render the window picture with the texture superimposed into the video memory to obtain video memory picture data; and a first data submitting unit adapted to submit the video memory picture data to a window manager for rendering. 7.A computing device, comprising: at least one processor and a memory having program instructions stored therein; when the program instructions are read and executed by the processor, the computing device is caused to perform the window rendering acceleration method of any one of claims 1-5. 8.A readable storage medium having program instructions stored therein, when the program instructions are read and executed by a computing device, the computing device is caused to perform the window rendering acceleration method of any one of claims 1-5. ​

Images