GPU hardware and software module and GPU graphics display buffer exchange synchronization method

Abstract

The invention provides a GPU software and hardware module and a GPU graphic display buffer exchange synchronization method, performs GPU software and hardware module framework design, software and hardware data flow design, user state driver layered architecture design, specifically the design and optimization of the EGL eglSwapBuffer API interface execution process, covers a series of interactive processes from application layer API call to user state driver response to the API, and then to hardware support for software behavior, etc., realizes GPU rendering event preprocessing and software and hardware synchronization by using an interrupt mechanism, and then completes the double buffer or multi-buffer exchange of the buffer currently being displayed and the buffer that has been rendered by the GPU.

Description

Technical Field

[0001] The present invention relates to the field of GPU graphics rendering, and more specifically, to a GPU software and hardware module and a GPU graphics display buffer exchange synchronization method. Background Art

[0002] In the field of computer graphics processing, double buffering or multi-buffering technology is usually used to achieve smooth graphics display effects. This technology uses two or more independent buffers to store graphics data, one of which (called the front buffer) is used to store the graphics data currently displayed on the screen, while the other one or more buffers are used to store the next frame of graphics data to be displayed (called the back buffer). When the graphics processor GPU (Graphics Processing Unit) completes the drawing of a frame of graphics, the system will perform a display buffer swap operation.

[0003] The core of the display buffer swap operation is to swap the buffer currently used for display with the buffer used to draw the next frame of graphics data. This process is usually controlled by vertical synchronization technology to ensure that the swap operation is performed during the vertical retrace period of the display to avoid image tearing.

[0004] A similar technical solution is the patent number CN116909511A "Method, device and storage medium for improving GPU double buffer display efficiency", which discloses a method for improving GPU double buffer display efficiency, the method comprising: storing the rendering result of the current frame in a first buffer; exchanging the first buffer with the second buffer; during the exchange process, performing a memory copy through the NENO instruction of the ARM platform to copy the rendering result of the current frame to the second buffer; sending the rendering result in the second buffer to the display to be displayed on the screen.

[0005] The above patent has the following defects:

[0006] (1) This patent is closely related to the ARM platform and relies on the ARM instruction set. Therefore, it cannot be implemented on non-ARM architecture platforms, such as RISC-V architecture or fully forward-developed independent architecture platforms.

[0007] (2) During the data exchange process, the NENO instruction set of the ARM platform is used to perform memory copying to copy the rendering result of the current frame to the second buffer. This process involves a memory copy operation. Summary of the invention

[0008] The present invention aims at the technical problems existing in the prior art and provides a GPU software and hardware module and a GPU graphic display buffer exchange synchronization method to solve the problem of redundant operations caused by a single platform and data copying.

[0009] According to a first aspect of the present invention, a GPU software and hardware module is provided, comprising:

[0010] The GPU software and hardware module includes an application program, a user-mode driver, a kernel-mode driver, and a hardware module. The application program includes an OpenGL API interface and an EGL API interface. The hardware module includes a GPU unit and a display controller.

[0011] The application will initialize and configure the display device by calling the EGL API interface, and initialize and configure the GPU unit through the OpenGL API interface;

[0012] The application sends a task request to the user-state driver through the OpenGL API interface. After receiving the task request, the user-state driver checks the user input data and converts the user input data into internal data for processing and integration, and sends the data and commands to the kernel-state driver in the form of a task package, which includes a data packet and a command packet;

[0013] After the kernel-mode driver obtains the task packet of the user-mode driver, it puts the data packet into the memory or the video memory, and passes the command packet to the hardware module. The GPU unit in the hardware module renders and draws the data packet, and puts the rendering result into the buffer;

[0014] The user-state driver monitors the task asynchronously or synchronously while issuing the task request. When the GPU unit completes the rendering task and puts the rendering result into the buffer, it synchronizes it to the kernel-state driver in the form of an interrupt. The kernel-state driver reports the interrupt event to the user-state driver in the form of an event trigger mechanism.

[0015] After the user-mode driver obtains the completion status of the GPU unit rendering and drawing task in an asynchronous or synchronous waiting manner, if the application calls the eglSwapBuffer API interface at the same time after initiating the GPU rendering and drawing task, the display device is controlled in the implementation of eglSwapBuffer to exchange the buffer currently being displayed with the buffer that has been rendered and drawn by the GPU unit.

[0016] Based on the above technical solution, the present invention can also make the following improvements.

[0017] Optionally, the user state driver includes an API interface layer, a state processing layer, a device driver layer and a system abstraction layer;

[0018] The API interface layer is used to receive the task request transmitted by the application program, check and structure the user input data according to the task request, and convert the user input data into internal data and submit it to the state processing layer;

[0019] The state processing layer is used to process the internal data;

[0020] The device driver layer is used to integrate and package the processed internal data, and send the data and commands to the kernel state driver in the form of task packages;

[0021] The system abstraction layer is used to achieve compatibility with multiple system platforms.

[0022] Optionally, the kernel-mode driver includes a GPU driver and a display controller driver, which are used to drive the GPU unit and the display controller in the hardware module respectively.

[0023] According to a second aspect of the present invention, a GPU graphics display buffer exchange synchronization method is provided, comprising:

[0024] When the user-mode driver receives the task request of the EGL API interface, it initializes the display device, creates a screen object, prepares one or more frame buffers in the screen object and sets the initial address scanned by the display controller, the initial address scanned by the display controller is the buffer currently being displayed; the application controls the GPU unit through the OpenGL API interface;

[0025] When the user-state driver receives a task request from the OpenGL API interface, it creates a context object and sets it as the current context through the EGL API interface. The user-state driver processes and packages the user input data in the current context and submits it to the kernel-state driver in the form of a task package. The command processed by the user-state driver carries a GPU rendering destination address, which points to a buffer that has been rendered by the GPU unit, and the GPU rendering destination address is different from the initial address scanned by the display controller;

[0026] After the user-mode driver sends the rendering task to the kernel-mode driver, the kernel-mode driver schedules the rendering task and passes the rendering task to the GPU unit in the hardware module for rendering, and places the rendering result in a buffer corresponding to the GPU rendering destination address;

[0027] When the user-state driver sends the rendering task to the kernel-state driver, the user-state driver will enter an asynchronous waiting state or a synchronous waiting state until the GPU unit rendering task is completed; after the GPU unit rendering is completed and the rendering result is placed in the buffer, it will be synchronized to the kernel-state driver in an interrupt manner;

[0028] The kernel-state driver receives an interrupt event, and the user-state driver detects that the event is triggered. If the application initiates an eglSwapBuffer API request, the user-state driver uses the register read and write interface provided by the kernel-state driver to write the GPU rendering destination address into the register as the frame buffer address that the display controller needs to scan, thereby realizing the switching between the buffer that the GPU unit has rendered and the buffer that is currently being displayed.

[0029] Optionally, two frame buffers are prepared in the screen object, and the initial address scanned by the display controller is set to point to the second buffer, and the GPU rendering destination address carried in the command processed by the user mode driver points to the first buffer.

[0030] Optionally, implement switching between the buffer that has been rendered by the GPU unit and the buffer that is currently being displayed, including:

[0031] During the first frame rendering process, the GPU unit renders the graphic data into the first buffer, and the display controller scans the content in the second buffer. When the hardware module reports an interrupt event indicating that the rendering is completed, the user-mode driver receives the interrupt event and switches the addresses of the first buffer and the second buffer.

[0032] During the second frame rendering process, the GPU unit renders the graphic data into the second buffer, the display controller scans the content in the first buffer, and when the hardware module reports an interrupt event indicating that the rendering is completed, the user-mode driver receives the interrupt event and switches the addresses of the second buffer and the first buffer;

[0033] During the third frame rendering process, the GPU unit renders the graphic data into the first buffer, and the display controller scans the content in the second buffer. When the hardware module reports an interrupt event indicating that the rendering is completed, the user-mode driver receives the interrupt event and switches the addresses of the first buffer and the second buffer.

[0034] During the rendering of the fourth frame, the GPU unit renders the graphic data into the second buffer, the display controller scans the content in the first buffer, and when the hardware module reports an interrupt event indicating that the rendering is completed, the user-mode driver receives the interrupt event and switches the addresses of the second buffer and the first buffer;

[0035] In each rendering process, the address of the first buffer and the address of the second buffer are switched back and forth in a cycle.

[0036] Optionally, three frame buffers are prepared in the screen object, wherein the display controller owns a first buffer, the GPU unit owns a second buffer, and the CPU owns a third buffer;

[0037] The switching between the buffer that has been rendered by the GPU unit and the buffer that is currently being displayed includes:

[0038] When the GPU unit is drawing graphics in the second buffer, the CPU can still send graphics drawing commands for the third buffer to the GPU unit. When the user-mode driver on the CPU receives the GPU unit rendering completion event, it swaps the first buffer and the second buffer where the rendering is completed to complete one frame of display.

[0039] When the rendering of the third buffer is completed, the third buffer and the first buffer are exchanged.

[0040] The present invention provides a GPU software and hardware module and a GPU graphics display buffer exchange synchronization method, which performs GPU software and hardware module framework design, software and hardware data flow design, and user-mode driver layered architecture design, specifically the design and optimization of the EGLeglSwapBuffer API interface execution process, covering a series of interactive processes from application layer API calls to user-mode driver responses to the API, and then to hardware support for software behavior, etc., and uses an interrupt mechanism to implement GPU rendering event preprocessing and software and hardware synchronization, and then completes the exchange of double buffers or multiple buffers between a buffer currently being displayed and a buffer that has been rendered by the GPU. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A GPU hardware and software module framework diagram provided by the present invention;

[0042] Figure 2 A flowchart of a GPU graphics display buffer exchange synchronization method provided by the present invention;

[0043] Figure 3 It is the interaction flow chart in GPU software and hardware modules;

[0044] Figure 4 Swap control flow graph for buffers;

[0045] Figure 5 Flowchart for buffer exchange logic;

[0046] Figure 6 Schematic diagram of the switching operation of the double buffer;

[0047] Figure 7 Schematic diagram of the switching operation of three buffers. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. In addition, the technical features in the various embodiments or single embodiments provided by the present invention can be arbitrarily combined with each other to form a feasible technical solution. This combination is not subject to the constraints of the sequence of steps and / or the structural composition mode, but must be based on the ability of ordinary technicians in this field to achieve. When the combination of technical solutions is contradictory or cannot be achieved, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0049] As rendering effects continue to improve, the rendering pressure faced by graphics processors is also increasing. In this case, it is difficult for graphics processors to always maintain a faster speed than display controllers. In addition, the display controller also draws pixels one by one in the order from left to right and from top to bottom during the display process. Therefore, it is inevitable that users will encounter flickering, screen tearing and other phenomena during the display process. In order to solve these problems, it is an effective solution to put the rendering process into invisible video memory using double buffering or triple buffering technology. In addition, it can also be achieved through buffer replication, hardware-based buffer switching and other methods. However, in the prototype verification stage of field programmable gate arrays (FPGAs), it usually only relies on the display function of the FPGA itself. In this context, a cost-effective, universally applicable and hardware-independent problem-solving solution is proposed, namely the solution of the present invention. Since the solution does not depend on a specific hardware platform, the method is also applicable to any other computer platform.

[0050] Figure 1 The present invention provides a GPU hardware and software module, in which the GPU unit, video memory and display controller are designed as independent modules, which interact through a bus. This design allows efficient data transmission between modules to ensure smooth and efficient graphics rendering. The configuration of the video memory is to meet the data storage needs of the CPU and GPU when executing applications. Video memory, also known as frame buffer, is a memory area specifically used to store rendered image data. It is usually divided into different buffers, such as front buffer, back buffer, etc., to support various operations in the graphics rendering process.

[0051] The front buffer is usually used to store image data that will be displayed on the screen, while the image data that the current GPU is drawing is stored in the back buffer. This double buffering technology can effectively avoid flickering during image rendering and improve display quality. In some cases, the video memory can also be separated from the system memory to optimize performance and management. For example, an embodiment of the present invention adopts a design in which the system memory is separated from the video memory, so that memory resources can be managed more flexibly and the efficiency of graphics rendering can be improved.

[0052] Figure 1 A framework diagram of a GPU software and hardware module is shown. The GPU software and hardware module includes an application, a user-mode driver, a kernel-mode driver, and a hardware module. The application includes an OpenGL API interface and an EGL API interface, and the hardware module includes a GPU unit and a display controller.

[0053] In an application, when the application calls the OpenGL and EGL APIs, the user-state driver implements API capability support and integrates data, and connects with the kernel-state driver to implement abstract control of the underlying devices. The kernel-state driver implements the underlying control of the hardware devices and provides the basic device control interface to the user state.

[0054] The application initializes and configures the display device by calling the EGL API interface, and initializes and configures the GPU device through the OpenGL API interface.

[0055] The application sends a task request to the user-state driver through the OpenGL API interface. After receiving the task request, the user-state driver checks the user input data and converts the user input data into internal data for processing and integration, and sends the data and commands to the kernel-state driver in the form of a task package. The task package includes a data packet and a command packet.

[0056] After the kernel-mode driver obtains the task packet from the user-mode driver, it puts the data packet into the memory or video memory, passes the command packet to the hardware module, and the GPU unit in the hardware module renders and draws the data packet and puts the rendering result into the buffer, which is the buffer where the GPU has completed rendering.

[0057] The user-state driver monitors the rendering task asynchronously or synchronously while issuing task requests. When the GPU unit completes the rendering task and puts the rendering result into the buffer, it synchronizes it to the kernel-state driver in an interrupt manner. The kernel-state driver reports the interrupt event to the user-state driver using an event trigger mechanism.

[0058] After the user-mode driver obtains the completion status of the GPU unit rendering and drawing task in an asynchronous or synchronous waiting manner, if the application calls the eglSwapBuffer API at the same time after initiating the GPU rendering and drawing task, indicating that the time for buffer exchange has arrived, the display device is controlled in the implementation of eglSwapBuffer to exchange the buffer currently being displayed with the buffer that has been rendered and drawn by the GPU unit. In this way, the buffer originally waiting in the background becomes the new display content, and the buffer previously used for display is transferred to the background for the GPU to continue drawing, so as to maximize the efficiency of the GPU and screen display.

[0059] Among them, EGL, the full name is OpenGL ES Native Platform Graphics Library, which can also be understood as Kronos Native Platform Graphics Interface, or Embedded-System Graphics Library. The main function of EGL is to serve as an interface for docking and interaction between OpenGL and the local window system.

[0060] This process not only involves the exchange of graphics data, but also includes the reconfiguration of the graphics rendering pipeline to ensure that the new buffer can correctly receive and render graphics data. The entire data flow process requires a high degree of accuracy and synchronization to avoid screen tearing or display errors.

[0061] like Figure 1 As shown, the user-mode driver includes an API interface layer, a state processing layer, a device driver layer, and a system abstraction layer;

[0062] The API interface layer is used to receive task requests transmitted by the application program, check and structure the user input data according to the task request, and convert the user input data into internal data and submit it to the state processing layer;

[0063] A state processing layer, used for processing the internal data;

[0064] The device driver layer is used to integrate and package the processed internal data and send the data and commands to the kernel-mode driver in the form of task packages;

[0065] The system abstraction layer is used to achieve compatibility with multiple system platforms, so that the GPU software and hardware modules provided by the present invention can be applied to multiple platforms.

[0066] Among them, the kernel state driver includes the GPU driver and the display controller driver, which are used to drive the GPU unit and the display controller in the hardware module respectively, drive the GPU unit to perform rendering tasks, and the display controller controls the display device for display.

[0067] Due to the complexity of the exchange synchronization method, the present invention also proposes a GPU graphics display buffer exchange synchronization method, which is applied to the GPU software and hardware modules. Through the fine control of the driver and the rational use of the interrupt mechanism, the efficiency and stability of the frame buffer switching are ensured, thereby providing the application with a smooth and stable graphics display effect.

[0068] See also Figure 2 and Figure 3 The GPU graphics display buffer exchange synchronization method uses an event trigger mechanism, so that the user state can perceive the change of the rendering state, and directly initialize and scan the address of the hardware display module through the mmap function method. It should be noted that the GPU rendering address is always different from the display controller's scan address. The steps of the GPU graphics display buffer exchange synchronization method include:

[0069] Step 1: When the user-mode driver receives a task request from the EGL API interface, it initializes the display device, creates a screen object, prepares one or more frame buffers in the screen object, and sets the initial address scanned by the display controller. The initial address scanned by the display controller is the buffer currently being displayed; the application controls the GPU device through OpenGL-related APIs (such as glClear, glDrawArrays, glFlush, etc.).

[0070] Step 2. When the user-state driver receives the task request of the OpenGL API interface, it creates a context object and sets it as the current context through the EGL API interface. The subsequent process will be based on the current context. The user-state driver processes and packages the user input data in the current context, and submits the application's task request to the kernel driver in the form of a task. The glClear or glDrawArrays API commands processed by the user-state driver will carry the GPU rendering destination address. The GPU rendering destination address points to the buffer that the GPU unit has already rendered. The GPU rendering destination address is different from the initial address scanned by the display controller.

[0071] Step 3: After the user-mode driver sends the rendering task to the kernel-mode driver, the kernel-mode driver schedules the rendering task and passes the rendering task to the GPU unit in the hardware module for rendering, and places the rendering result in a buffer corresponding to the GPU rendering destination address.

[0072] Step 4: When the user-state driver sends the rendering task to the kernel-state driver, the user-state driver will enter an asynchronous waiting or synchronous waiting state until the GPU unit rendering task is completed. If it is an asynchronous waiting state, pre-waiting can be performed, that is, the task of interest is monitored before the task is sent. This can effectively solve the problem of event state loss caused by the GPU rendering task being completed when the waiting behavior is not initialized when the GPU rendering speed is fast. After the GPU unit rendering is completed and the rendering result is placed in the buffer, it is synchronized to the kernel-state driver in an interrupt manner.

[0073] Step 5: The kernel-mode driver receives an interrupt event, and the user-mode driver detects that the event is triggered. If the application initiates an eglSwapBuffer API request, the user-mode driver can use the mmap method or the register read / write interface provided by the kernel-mode driver to write the GPU rendering destination address into the register as the frame buffer address that the display controller needs to scan, thereby switching between the buffer that the GPU unit has rendered and the buffer that is currently being displayed. For example, if address 1 (pointing to the first buffer) is set during the initialization phase, then after receiving the interrupt event, the address will switch to the second buffer. The user-mode driver device driver layer needs to initialize the display controller and exchange the display controller with the GPU task delivery process to ensure the consistency of the GPU and display controller in reading and writing the buffer.

[0074] Figure 4 This is the buffer exchange control flow chart, and its detailed operation process is as follows:

[0075] When the user-state driver receives a task request from the EGL API interface, it initializes the display device, creates a screen object, prepares one or more frame buffers in the screen object, and initializes the initial address scanned by the display controller as the second buffer. When the user-state driver receives a task request from the OpenGL API interface, it creates a context object, initializes the event mechanism to wait for events, and sends the address of the first buffer as the GPU rendering destination address.

[0076] When the GPU unit rendering task is completed, it is synchronized to the kernel state driver in an interrupt mode. The kernel state driver receives the interrupt event, the user state driver detects that the event is triggered, and the application initiates the eglSwapBuffer API request to perform buffer swap logic processing, and sends the address of the first buffer as the scanning address of the display controller to achieve switching between the first buffer and the second buffer.

[0077] According to the specific application scenario:

[0078] During the first frame rendering process, the GPU renders the graphics data into the first buffer, and the display controller scans the contents of the second buffer. When the hardware reports an interrupt event indicating that the rendering is complete, the user-mode driver receives the event and immediately switches the frame buffer address.

[0079] During the second frame rendering process, the GPU renders the graphics data into the second buffer, and the display controller scans the contents of the first buffer. Similarly, when the hardware reports an interrupt event that the rendering is completed, the user-mode driver receives this event and switches the frame buffer address again.

[0080] During the third frame rendering process, the GPU renders the graphics data into the first buffer, and the display controller scans the contents of the second buffer. When the hardware reports an interrupt event indicating that the rendering is complete, the user-mode driver receives this event and switches the frame buffer address again.

[0081] During the rendering of the fourth frame, the GPU renders the graphics data into the second buffer (back buffer), and the display controller scans the contents of the first buffer (front buffer). When the hardware reports an interrupt event that the rendering is complete, the user-mode driver receives this event and switches the frame buffer address again.

[0082] This process will continue to form a loop. The core logic is to ensure that the frame buffer addresses obtained by the GPU and the display controller are always inconsistent. When the rendering task is heavy, in order to provide the GPU with more rendering time, a third buffer or more buffers can be introduced. This can avoid problems such as screen tearing caused by slow GPU rendering when the rendering task is heavy. The core exchange process is as follows Figure 5 , the application executes EGLeglSwapBuffers to drive the GPU and display controller to schedule buffers and complete the buffer addresses. The swap can realize the switching of the front buffer (storing the image data currently being displayed) and the back buffer (storing the image data rendered and drawn by the GPU, that is, the image data to be displayed).

[0083] The schematic diagram of the double buffer switching operation can be found in Figure 6 , the schematic diagram of the switching operation of the three buffers can be seen in Figure 7 .

[0084] The switching principle of the double buffer is as follows: the GPU renders the graphics data into the first buffer, and the display controller scans the contents of the second buffer. When the hardware reports an interrupt event that the rendering is completed, the user-mode driver receives this event and immediately switches the frame buffer address.

[0085] When three frame buffers are prepared in the screen object, the GPU unit owns the first buffer ( Figure 7 The CPU has a second buffer ( Figure 7 The display controller has a third buffer (i.e. Figure 7 in the front buffer).

[0086] The switching principle of the three buffers is: when the GPU unit is drawing graphics in the first buffer, the CPU can still send graphics drawing commands for the second buffer to the GPU unit. When the user-mode driver on the CPU receives the GPU unit rendering completion event, it exchanges the addresses of the first buffer and the third buffer that have been rendered to complete one frame of display; when the second buffer is rendered, the addresses of the second buffer and the third buffer are exchanged again, and the exchange between the first buffer, the second buffer and the third buffer is realized in a round-robin manner.

[0087] The switching process of the three buffers can be seen in Table 1.

[0088] Table 1

[0089] ,

[0090] A GPU software and hardware module and a GPU graphics display buffer exchange synchronization method provided by an embodiment of the present invention have the following advantages:

[0091] (1) In the GPU driver, effective control of the display controller can be achieved through simple register configuration operations. This control method can be based on interrupt triggering or register polling to complete the switching of the buffer, thereby ensuring that the synchronization problem between the GPU and the display controller is properly resolved.

[0092] (2) The present invention controls the display controller through a user-mode software driver to complete the design of buffer switching, and can implement the function of eglSwapBuffers at a minimum cost, greatly reducing software and hardware overhead, providing a simple and easy-to-implement technical solution, improving the display effect, and reducing the complexity of development and implementation.

[0093] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0094] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0095] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0096] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0097] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0098] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0099] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A GPU hardware and software module, characterized in that: The GPU software and hardware module includes an application program, a user-mode driver, a kernel-mode driver, and a hardware module. The application program includes an OpenGL API interface and an EGL API interface. The hardware module includes a GPU unit and a display controller. The application will initialize and configure the display device by calling the EGL API interface, and initialize and configure the GPU unit through the OpenGL API interface; The application sends a task request to the user-state driver through the OpenGL API interface. After receiving the task request, the user-state driver checks the user input data and converts the user input data into internal data for processing and integration, and sends the data and commands to the kernel-state driver in the form of a task package, which includes a data packet and a command packet; After the kernel-mode driver obtains the task packet of the user-mode driver, it puts the data packet into the memory or the video memory, and passes the command packet to the hardware module. The GPU unit in the hardware module renders and draws the data packet, and puts the rendering result into the buffer; The user-state driver monitors the task asynchronously or synchronously while issuing the task request. When the GPU unit completes the rendering task and puts the rendering result into the buffer, it synchronizes it to the kernel-state driver in the form of an interrupt. The kernel-state driver reports the interrupt event to the user-state driver in the form of an event trigger mechanism. After the user-mode driver obtains the completion status of the GPU unit rendering and drawing task in an asynchronous or synchronous waiting manner, if the application calls the eglSwapBuffer API interface at the same time after initiating the GPU rendering and drawing task, the display device is controlled in the implementation of eglSwapBuffer to exchange the buffer currently being displayed with the buffer that has been rendered and drawn by the GPU unit.

2. The GPU software and hardware module according to claim 1, characterized in that: The user state driver includes an API interface layer, a state processing layer, a device driver layer and a system abstraction layer; The API interface layer is used to receive the task request transmitted by the application program, check and structure the user input data according to the task request, and convert the user input data into internal data and submit it to the state processing layer; The state processing layer is used to process the internal data; The device driver layer is used to integrate and package the processed internal data, and send the data and commands to the kernel state driver in the form of task packages; The system abstraction layer is used to achieve compatibility with multiple system platforms.

3. The GPU hardware and software module according to claim 2, characterized in that: The kernel state driver includes a GPU driver and a display controller driver, which are used to drive the GPU unit and the display controller in the hardware module respectively.

4. A GPU graphics display buffer exchange synchronization method, applied to the GPU device of claim 1, characterized in that: The method comprises: When the user-mode driver receives the task request of the EGL API interface, it initializes the display device, creates a screen object, prepares one or more frame buffers in the screen object and sets the initial address scanned by the display controller, the initial address scanned by the display controller is the buffer currently being displayed; the application controls the GPU unit through the OpenGL API interface; When the user-state driver receives a task request from the OpenGL API interface, it creates a context object and sets it as the current context through the EGL API interface. The user-state driver processes and packages the user input data in the current context and submits it to the kernel-state driver in the form of a task package. The command processed by the user-state driver carries a GPU rendering destination address, which points to a buffer that has been rendered by the GPU unit, and the GPU rendering destination address is different from the initial address scanned by the display controller; After the user-mode driver sends the rendering task to the kernel-mode driver, the kernel-mode driver schedules the rendering task and passes the rendering task to the GPU unit in the hardware module for rendering, and places the rendering result in a buffer corresponding to the GPU rendering destination address; When the user-mode driver sends the rendering task to the kernel-mode driver, the user-mode driver will enter an asynchronous waiting state or a synchronous waiting state until the GPU unit rendering task is completed; after the GPU unit rendering is completed and the rendering result is placed in the buffer, it will be synchronized to the kernel-mode driver in an interrupt manner; The kernel-state driver receives an interrupt event, and the user-state driver detects that the event is triggered. If the application initiates an eglSwapBuffer API request, the user-state driver uses the register read and write interface provided by the kernel-state driver to write the GPU rendering destination address into the register as the frame buffer address that the display controller needs to scan, thereby realizing the switching between the buffer that the GPU unit has rendered and the buffer that is currently being displayed.

5. The GPU graphics display buffer exchange synchronization method according to claim 4, characterized in that: Two frame buffers are prepared in the screen object, and the initial address scanned by the display controller is set to point to the second buffer, and the GPU rendering destination address carried in the command processed by the user mode driver points to the first buffer.

6. The GPU graphics display buffer exchange synchronization method according to claim 5, characterized in that: Implement the switching between the buffer that has been rendered by the GPU unit and the buffer that is currently being displayed, including: During the first frame rendering process, the GPU unit renders the graphic data into the first buffer, and the display controller scans the content in the second buffer. When the hardware module reports an interrupt event indicating that the rendering is completed, the user-mode driver receives the interrupt event and switches the addresses of the first buffer and the second buffer. During the second frame rendering process, the GPU unit renders the graphic data into the second buffer, the display controller scans the content in the first buffer, and when the hardware module reports an interrupt event indicating that the rendering is completed, the user-mode driver receives the interrupt event and switches the addresses of the second buffer and the first buffer; During the third frame rendering process, the GPU unit renders the graphic data into the first buffer, and the display controller scans the content in the second buffer. When the hardware module reports an interrupt event indicating that the rendering is completed, the user-mode driver receives the interrupt event and switches the addresses of the first buffer and the second buffer. During the rendering of the fourth frame, the GPU unit renders the graphic data into the second buffer, the display controller scans the content in the first buffer, and when the hardware module reports an interrupt event indicating that the rendering is completed, the user-mode driver receives the interrupt event and switches the addresses of the second buffer and the first buffer; In each rendering process, the address of the first buffer and the address of the second buffer are switched back and forth in a cycle.

7. The GPU graphics display buffer exchange synchronization method according to claim 4, characterized in that: Three frame buffers are prepared in the screen object, wherein the display controller owns a first buffer, the GPU unit owns a second buffer, and the CPU owns a third buffer; The switching between the buffer that has been rendered by the GPU unit and the buffer that is currently being displayed includes: When the GPU unit is drawing graphics in the second buffer, the CPU can still send graphics drawing commands for the third buffer to the GPU unit. When the user-mode driver on the CPU receives the GPU unit rendering completion event, it swaps the first buffer and the second buffer where the rendering is completed to complete one frame of display. When the rendering of the third buffer is completed, the third buffer and the first buffer are exchanged.

Images