An implementation method and system for applying Qt-based graphic rendering to DRM in X11

Abstract

The application discloses an implementation method and system of graphic rendering to DRM applied to X11 and based on Qt, and the method comprises the following steps: creating and initializing a layer management and synthesis engine module OSDRenderCore, requesting registration of a global shared memory to X11 after successful initialization; creating a library OSDRenderBase for an application window and requesting allocation of a sub-shared memory in the global shared memory after receiving a display request of the application window; in step 3, the library OSDRenderBase renders data of the application window to a QPainter class under Qt through an interface render of a qwidget class of Qt, then copies the data of the application window on the QPainter class to the sub-shared memory corresponding to the application window, and the layer management and synthesis engine module OSDRenderCore takes the data of the application window from the sub-shared memory and sends the taken data to DRM. The application avoids that the application window is shielded, and reduces the development threshold.

Description

Technical Field

[0001] This invention relates to the field of graphics rendering and display technology, specifically a method and system for implementing graphics rendering to DRM based on Qt and applied to X11. Background Technology

[0002] Currently, many embedded platforms, especially modern high-performance embedded platforms (such as embedded devices using the RK3588 chip), typically run standard X11 desktop operating systems (such as Ubuntu). Using X11 results in a unique hybrid graphics architecture. On one hand, the standard desktop operating system running X11 has a complete X11 desktop environment, allowing standard applications to run; on the other hand, to achieve optimal view encoding / decoding and rendering performance, the hardware relies on the GPU for both. This causes encoding / decoding and rendering to bypass X11 and instead directly output content to the hardware overlay via DRM (Direct Rendering Manager). Because this direct output to the overlay via DRM has the highest display priority, it completely obscures the entire X11 desktop, making all standard application windows invisible, appearing "not displayed." Therefore, how to still display application windows on a screen covered by the DRM hardware overlay becomes a technical problem that needs to be solved.

[0003] In existing technologies, two main approaches are generally used to address this technical problem: First, abandoning X11: Developing audio and video applications, including views, entirely using hardware-based DRM / KMS. With this approach, developers cannot directly use the interface; they need to write business-level UI elements using the underlying DRM / KMS interface. This development is extremely difficult, creating aesthetically pleasing interfaces is very challenging, and existing related code (such as Qt's QWidet code) cannot be reused. Second, using the Wayland framework (also known as the Wayland protocol): With this approach, the Wayland framework's synthesizer can cooperate with DRM, allowing Qt's interface to work well with and overlay DRM. This maintains the performance of chip GPU acceleration while retaining the convenience of Qt interface development. However, many domestic operating systems and third-party Linux distributions are not fully compatible with the Wayland framework, and many third-party software ecosystems do not support Wayland. Furthermore, migrating applications from X11 to the Wayland framework requires significant adaptation work.

[0004] In summary, in order to achieve the implementation of UI-based development and abandon the Wayland framework, a new technology is needed that can render graphics to DRM without obscuring the application window. Summary of the Invention

[0005] In view of the shortcomings of the prior art, the purpose of this invention is to provide a method and system for implementing graphics rendering to DRM based on Qt and applied to X11, which can solve the problems described in the background art.

[0006] The technical solution to achieve the objective of this invention is: a method for implementing graphics rendering to DRM based on Qt and applied to X11, comprising the following steps: Step 1: Create and initialize the layer management and compositing engine module OSDRenderCore. After successful initialization, request X11 to register a global shared memory. The global shared memory is used for the transfer of windows of all applications. The application windows are windows built based on Qt. Step 2: Upon receiving a request to display the application window, create a library OSDRenderBase for the application window. The library OSDRenderBase is based on Qt's QtWidget class. Request the allocation of a sub-shared memory within the global shared memory. The sub-shared memory is used to share the application window's data within the library OSDRenderBase, and to configure rendering flags for the sub-shared memory. Step 3: The OSDRenderBase library first renders the application window's data to the QPainter class in Qt using the render interface of the Qwidget class. Then, it copies the application window's data from the QPainter class to the shared memory corresponding to the application window. The layer management and compositing engine module OSDRenderCore retrieves data from the application window in the sub-shared memory and sends the retrieved data to DRM. DRM then renders the received data, thus completing the entire rendering process.

[0007] Furthermore, the layer management and compositing engine module OSDRenderCore is also used to listen for the registration of windows of various applications, and when there are windows with the same combined name, the old application window is deleted and replaced with the new application window.

[0008] Furthermore, the OSDRenderBase library is also used to create a library name consisting of a unique identifier, which includes the name and the id. In other words, the combination of the name and the id constitutes the library name.

[0009] Furthermore, the library OSDRenderBase initializes the interface to implement the OSD layer desired by the input window name and application window, thereby creating the corresponding sub-shared memory and registering the window name and application window with the layer management and compositing engine module OSDRenderCore.

[0010] Furthermore, the OSDRenderBase library also includes a startRender flag and a stopRender flag.

[0011] Furthermore, when there are multiple application windows that need to be rendered, the layer management and compositing engine module OSDRenderCore retrieves the data of each application window from the corresponding sub-shared memory, and combines them into an image data that is the same size as the target screen and is transparent and superimposed on the target screen. The image data is then sent to DRM, and DRM renders the received graphic data to render an image.

[0012] An implementation system for graphics rendering to DRM based on Qt and applied to X11, comprising: The application window module is used to create application windows based on Qt, and configures a corresponding OSDRenderBase library for each application window. The OSDRenderBase library is used to request the allocation of a sub-shared memory, and to render the data of the application window to the QPainter class in Qt through the render interface of the Qwidget class, and then copy the data of the application window in the QPainter class to the sub-shared memory corresponding to the application window. The sub-shared memory is used for data of the application window within the shared library OSDRenderBase, and for configuring rendering flags for the sub-shared memory; The shared memory management module is used to create a global shared memory based on the received shared memory creation request, and to configure a corresponding sub-shared memory for each library OSDRenderBase within the global shared memory; The graphics management and compositing engine module is used to initiate a shared memory creation request to register a global shared memory with X11; and to listen for the registration of windows of various applications. When a display request of an application window is heard, the module retrieves the application window's data from the sub-shared memory and sends the retrieved data to the DRM so that the DRM can render the received data, thereby completing the entire rendering process.

[0013] The beneficial effects of this invention are as follows: This invention breaks away from the existing DRM hardware overlay layer that completely obscures the X11 desktop, thus preventing application windows from being blocked. This allows users to see and operate various UI interfaces managed by this system while watching high-performance videos, greatly improving the user experience and functionality of the product. Furthermore, this invention does not require abandoning the familiar QWidget development model, nor does it require rewriting any existing UI code, lowering the development threshold. Attached Figure Description

[0014] Figure 1 This is a flowchart illustrating a preferred embodiment of the method of the present invention; Figure 2 This is a schematic diagram of the principle framework of the present invention; Figure 3 This is a schematic diagram of the framework structure of the system of the present invention. Detailed Implementation

[0015] The present invention will be further described below with reference to the accompanying drawings and specific embodiments: like Figures 1-2 As shown, a method for implementing graphics rendering to DRM based on Qt and applied to X11 includes the following steps: Step 1: Create and initialize the layer management and compositing engine module OSDRenderCore. After successful initialization, request to register a global shared memory with X11. The global shared memory is used for the transfer of windows of all applications. The application windows are windows built based on Qt.

[0016] The layer management and compositing engine module OSDRenderCore listens to the registration of windows for each application, and if there are windows with the same combined name, it deletes the old application window and replaces it with the new one.

[0017] Step 2: Upon receiving a display request for the application window, create a library OSDRenderBase for that application window. OSDRenderBase is based on Qt's QtWidget class. The application also submits its window name and desired OSD hierarchy to OSDRenderBase and requests allocation of a sub-shared memory within the global shared memory. This sub-shared memory is used to share the application window's data within OSDRenderBase. A rendering flag is configured for this sub-shared memory. When the rendering flag is enabled (set to true), it indicates that rendering of the application window's data within OSDRenderBase is currently required; when the rendering flag is disabled (set to false), it indicates that rendering of the application window's data within OSDRenderBase is not currently required.

[0018] For example, the OSDRenderBase library also creates a library name consisting of a unique identifier, which includes the name and the id. In other words, the combination of the name and the id constitutes the library name, so that each OSDRenderBase library and application window can be identified by having a unique library name.

[0019] Understandably, the OSDRenderBase library, through its initialization interface, allows users to input the window name and the desired OSD layer for the application window to create the corresponding sub-shared memory and register this information with the layer management and compositing engine module OSDRenderCore. If registration fails, it can re-apply every few seconds. Furthermore, by passing in the window pointer, rendering flags can be sent to the OSDRenderBase library of the Qt application window, enabling OSDRenderBase to access the corresponding Qt application window's data.

[0020] The OSDRenderBase library also has startRender and stopRender flags to determine when to start and stop rendering, so that the application window’s data can be copied to the child shared memory in real time according to the frame rate.

[0021] After the layer management and compositing engine module OSDRenderCore is initialized, a global shared memory is randomly registered. Then, when an application window needs to be displayed, a block of memory is allocated within the global shared memory as a sub-shared memory. This sub-shared memory is used to temporarily store the data of the application window that needs to be displayed, allowing the application window to modify and pass its own application window data within this sub-shared memory.

[0022] One method is to set the rendering frequency to detect the status and frequency of the rendering flag, thus determining whether the rendering flag is currently on or off. For example, setting the rendering frequency to 30 times / second, which is a rendering speed of 30 frames / second, will detect the rendering flag 30 times within 1 second.

[0023] When the application window needs to be rendered, its rendering flag will automatically turn on. When the rendering flag is detected as on, it means that copying the application window's data to its corresponding shared memory is permitted. In other words, if the rendering flag is off, rendering will not occur and the process will simply idle.

[0024] Step 3: The library OSDRenderBase, built based on the QtWidget class in Qt, first renders the application window's data onto the QPainter class in Qt through the render interface of the qwidget class in Qt. Then, it copies the application window's data from the QPainter class to the shared memory corresponding to the application window.

[0025] The layer management and compositing engine module OSDRenderCore retrieves data from the application window in the sub-shared memory and sends the retrieved data to DRM. DRM then renders the received data, thus completing the entire rendering process.

[0026] Understandably, the data of the application window can be copied to the corresponding sub-shared memory according to the rendering frequency. For example, if the rendering frequency is 30 frames / second, the data of the application window can be copied to the corresponding sub-shared memory at a frequency of 30 times per second.

[0027] For example, this also includes setting a startRender flag and a stopRender flag. The rendering of the application window's data to the QPainter class via the Qt qwidget class's render interface only begins when the startRender flag is enabled. And when the stopRender flag is enabled, the rendering of the application window's data to the QPainter class stops. In other words, by using the startRender and stopRender flags, the rendering of the application window's data to the QPainter class via the render interface is controlled.

[0028] For example, when there are multiple application windows that need to be rendered, the layer management and compositing engine module OSDRenderCore retrieves the data of each application window from their respective shared memory sub-memory, and combines them into a transparent image that is overlaid on the target screen. This image data is then sent to the DRM, which renders the received graphics data to produce an image. By repeating the above steps, images can be continuously rendered.

[0029] like Figure 3 As shown, the present invention also provides an implementation system for graphics rendering to DRM based on Qt and applied to X11, comprising: The application window module is used to create application windows based on Qt. For each application window, a corresponding OSDRenderBase library is configured. OSDRenderBase requests the allocation of a sub-shared memory and uses the `render` interface of Qt's `qwidget` class to render the application window's data onto Qt's `QPainter` class. Then, the application window's data from the `QPainter` class is copied to the corresponding sub-shared memory. This sub-shared memory is used to share the application window's data within the OSDRenderBase library and to configure rendering flags for that sub-shared memory.

[0030] Understandably, for each Qt application window (i.e., interface) used under X11, a corresponding library OSDRenderBase will be configured.

[0031] The shared memory management module is used to create a global shared memory based on the received shared memory creation request, and to configure a corresponding sub-shared memory for each library OSDRenderBase within the global shared memory.

[0032] The graphics management and compositing engine module is used to initiate a shared memory creation request to register a global shared memory with X11; and to listen for the registration of windows of various applications. When a display request of an application window is heard, the module retrieves the application window's data from the sub-shared memory and sends the retrieved data to the DRM so that the DRM can render the received data, thereby completing the entire rendering process.

[0033] The embodiments disclosed in this specification are merely illustrative of one aspect of the invention, and the scope of protection of the invention is not limited to these embodiments. Any other functionally equivalent embodiments fall within the scope of protection of the invention. Those skilled in the art can make various other corresponding changes and modifications based on the technical solutions and concepts described above, and all such changes and modifications should fall within the scope of protection of the claims of this invention.

Claims

1. A method for implementing graphics rendering to DRM based on Qt and applied to X11, characterized in that, Includes the following steps: Step 1: Create and initialize the layer management and compositing engine module OSDRenderCore. After successful initialization, request to register a global shared memory with X11. All application windows are transferred through the global shared memory. The application windows are windows built based on Qt. Step 2: Upon receiving a request to display the application window, create a library OSDRenderBase for the application window. The library OSDRenderBase is based on the Qt qwidget class. Request the allocation of a sub-shared memory within the global shared memory. The sub-shared memory is used to share the application window's data within the library OSDRenderBase, and to configure rendering flags for the sub-shared memory. Step 3: The OSDRenderBase library first renders the application window's data to the QPainter class in Qt using the render interface of the Qwidget class. Then, it copies the application window's data from the QPainter class to the shared memory corresponding to the application window. The layer management and compositing engine module OSDRenderCore retrieves data from the application window in the sub-shared memory and sends the retrieved data to the DRM. The DRM then renders the received data, thus completing the entire rendering process. The library OSDRenderBase initializes the OSD layer desired by the input window name and application window through an initialization interface, thereby creating the corresponding sub-shared memory and registering the window name and application window with the layer management and compositing engine module OSDRenderCore. When there are multiple application windows that need to be rendered, the layer management and compositing engine module OSDRenderCore retrieves the data of each application window from the corresponding shared memory, and combines them into an image data that is the same size as the target screen, transparent, and superimposed on the target screen. The image data is then sent to the DRM, and the DRM renders the received image data to produce an image.

2. The method for implementing graphics rendering to DRM based on Qt and applied to X11 according to claim 1, characterized in that, The layer management and compositing engine module OSDRenderCore is also used to listen for the registration of windows of various applications, and when a window with the same application window name exists, it deletes the old application window and replaces it with the new application window.

3. The method for implementing graphics rendering to DRM based on Qt and applied to X11 according to claim 1, characterized in that, The OSDRenderBase library is also used to create a library name consisting of a unique identifier, which includes the name and the id. In other words, the combination of the name and the id constitutes the library name.

4. The method for implementing graphics rendering to DRM based on Qt and applied to X11 according to claim 1, characterized in that, The OSDRenderBase library also includes a startRender flag and a stopRender flag.

5. A system for implementing graphics rendering to DRM based on Qt and applied to X11, characterized in that, include: The application window module is used to create application windows based on Qt. Upon receiving a display request for an application window, it configures a corresponding OSDRenderBase library for each window. The OSDRenderBase library is based on Qt's qwidget class. It requests the allocation of a sub-shared memory and uses the render interface of Qt's qwidget class to render the application window's data onto Qt's QPainter class. Then, it copies the application window's data from the QPainter class to the corresponding sub-shared memory for that application window. The sub-shared memory is used for data of the application window within the shared library OSDRenderBase, and for configuring rendering flags for the sub-shared memory; The shared memory management module is used to create a global shared memory based on the received shared memory creation request. All application windows are transferred through the global shared memory, and a corresponding sub-shared memory is configured for each library OSDRenderBase within the global shared memory. The graphics management and compositing engine module is used to initiate a shared memory creation request in order to request X11 to register a global shared memory; And it registers to listen for the windows of each application. When a display request for an application window is detected, it retrieves the application window's data from the sub-shared memory and sends the retrieved data to the DRM so that the DRM can render the received data, thereby completing the entire rendering process. The library OSDRenderBase initializes the OSD layer desired by the input window name and application window through an initialization interface, thereby creating the corresponding sub-shared memory and registering the window name and application window with the layer management and compositing engine module OSDRenderCore. When there are multiple application windows that need to be rendered, the layer management and compositing engine module OSDRenderCore retrieves the data of each application window from the corresponding shared memory, and combines them into an image data that is the same size as the target screen, transparent, and superimposed on the target screen. The image data is then sent to the DRM, and the DRM renders the received image data to produce an image.

Images