A system supporting Qt4 adaptation to Wayland and Wayland interaction method

Abstract

The present invention discloses a system supporting Qt4 to adapt to Wayland and a Wayland interaction method, which relates to the field of computer software technology, including a QtGui module, a Qt event processing module and a QWayland module; the QtGui module is used to draw a target interface for developers through the Qt4 native API; the Qt event processing module is used to encapsulate the events of the Wayland client through a QPA plug-in, and distribute the encapsulated Wayland client events to the event processing framework of Qt4; the QWayland module is used to process the information of the Wayland event, and call the Qt event processing module to encapsulate the events of the Wayland client. The present invention provides a more complete Wayland protocol, so that Qt4 can provide more interactive functions under Wayland, and improves the event processing mechanism for Wayland in Qt4, so that Qt4 can adapt to Wayland versions above 1.0, and provide developers with more friendly development tools.

Description

Technical Field

[0001] The present invention relates to the field of computer software technology, and in particular to a system supporting Qt4 to adapt to Wayland and a Wayland interaction method. Background Art

[0002] Qt4 is a cross-platform application development framework developed by the Qt Company (formerly Trolltech). It was released in 2005 and has become the basis for many desktop and embedded applications. Qt4 supports multiple operating systems, including Windows, macOS, and Linux, allowing developers to write code once and deploy it to multiple platforms. It provides a rich set of graphical user interface (GUI) components that developers can use to quickly build complex user interfaces. Qt4 is widely used in desktop applications, embedded systems, game development, and graphic design. Although it is still used in many projects, Qt5 and later versions have gradually replaced Qt4 over time.

[0003] Wayland is a modern display server protocol designed to replace the X Window System (commonly known as X11). It provides a more efficient, concise and intuitive way to handle graphical interfaces and input events. Here are some key features and advantages of Wayland: The design philosophy of Wayland is to minimize the middle layer so that the client can communicate directly with the display server to improve efficiency. Wayland can achieve smoother graphics performance and lower latency due to the reduction of protocol complexity and data transmission overhead; Wayland supports modern graphics technologies such as OpenGL es and Vulkan, which helps to develop high-performance graphics applications; Wayland is better than X11 in terms of security. Each application runs in its own sandbox, avoiding unnecessary interference between applications; Wayland represents a modern trend in graphics display protocols, committed to improving performance and security. With its increasing adoption rate, Wayland is becoming a new graphics infrastructure choice, especially in the Linux ecosystem.

[0004] However, Qt4 has limited support for Wayland. The version is still at Wayland 0.8, which has limited support for many functions, such as:

[0005] 1. Qt4 does not support the window control display function of Wayland1.0 and above versions.

[0006] 2. Qt4 does not support the mouse interaction function of Wayland1.0 and above.

[0007] 3. Qt4 does not support the keyboard interaction function of Wayland1.0 and above.

[0008] 4. Qt4 does not support the text display function of Wayland1.0 and above. Summary of the invention

[0009] In view of the above-mentioned deficiencies in the prior art, the present invention provides a system supporting Qt4 to adapt to Wayland and a Wayland interaction method to solve the problem that Qt4 is not compatible with Wayland versions above 1.0.

[0010] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is:

[0011] A system supporting Qt4 adaptation to Wayland is provided, which includes a QtGui module, a Qt event processing module and a QWayland module; wherein:

[0012] QtGui module, used to draw the developer's target interface through the Qt4 native API;

[0013] Qt event processing module, which is used to encapsulate the events of the wayland client through the qpa plug-in and distribute the encapsulated wayland client events to the event processing framework of qt4;

[0014] The QWayland module is loaded into the Qt4 development framework as a plug-in to process the information of the Wayland event and call the Qt event processing module to encapsulate the events of the Wayland client.

[0015] Furthermore, the QWayland module includes a QWaylanddisplay submodule, a QWaylandwindow submodule, a QWaylandInputDevice submodule, a QPlatformScreen submodule, and a QWaylandGLIntegration submodule; wherein:

[0016] The QWaylanddisplay submodule is used to establish and maintain the connection with the Wayland server. When an event sent from the Wayland server to the Wayland client is blocked on the Wayland client, it determines whether the current thread is processing an event. If so, the thread's task queue is immediately cleared and the event from the Wayland server is processed immediately. Otherwise, the event from the Wayland server is processed immediately. A window array and a surface array are maintained to manage the Wayland client with multiple windows and surfaces.

[0017] QWaylandwindow submodule, used to render and display the wayland client window;

[0018] The QWaylandInputDevice submodule is used to manage input devices, process events from input devices, call the Qt event processing module to encapsulate the wayland client input events as qt4 events, and forward them to the corresponding window; monitor the connection status and type changes of the input device, and issue the corresponding event type when the connection status and type of the input device change, so as to ensure that the application can respond to the changes of the input device in real time;

[0019] The QPlatformScreen submodule is integrated with the window management system of the operating system to provide screen information, screen attributes, support multi-screen display, and receive and process screen-related events;

[0020] The QWaylandGLIntegration submodule is used to manage the OpenGL es context; interact with the wayland server to handle the resources and status required in the OpenGL es rendering process; manage the frame buffer; receive events from windows and input devices and perform corresponding processing in the OpenGL es rendering loop; provide OpenGL es extensions; and provide logging support.

[0021] Furthermore, the QWaylandwindow submodule includes a QWaylandSurface component, a QWaylandBuffer component, a QWaylandShellSurface component, and a QWaylandShellIntegration component; wherein:

[0022] QWaylandSurface component, based on the wayland1.0 framework api, adds the interaction interface and buffer management interface with the wayland server, and modifies the interface for creating wl_surface or EGLSurface based on the wayland1.0 framework api; used to create and manage surfaces with wayland clients, encapsulate the interaction interface in wayland, provide interaction functions with the wayland server, and handle surface-related requests and events; maintain the state of the surface in the buffer, receive input events from the wayland server, and forward them to the corresponding window or component;

[0023] QWaylandBuffer component, by adding buffer and Wayland server notification mechanism, sets status symbols for buffer, used to transfer graphics data between application and Wayland server; performs surface association, associates graphics content to a specific surface; performs format support configuration; performs performance optimization through shared memory mechanism; handles the life cycle of buffer;

[0024] WaylandShellSurface component, used to create and manage surfaces that interact with the Wayland shell; manage user interface elements and provide support for developers to create traditional desktop windows; handle different states of windows, respond to, receive and process events from the Wayland client; provide decoration options, implement communication with the Wayland server by encapsulating the Wayland1.0 interface, and notify the decoration options through the server protocol to update the window position and size to ensure that the window meets the user's expectations when displayed;

[0025] The QWaylandShellIntegration component loads the xdg-shell protocol and wl-shell protocol as plug-ins based on the QWaylandShellintegration module of qt4 to create corresponding windows and surfaces.

[0026] Furthermore, in the QWaylandBuffer component, the graphics data passed includes data metadata related to the buffer, synchronization information data used to indicate when the buffer can be safely used or modified, color space information data, buffer format data, and additional user data added by the developer in the custom buffer view.

[0027] Furthermore, in the QWaylandwindow submodule, the specific method of rendering is:

[0028] Use EGL to create at least one buffer through the wayland client, specify the size and format of the buffer, and map the created buffer to the address space of the wayland client;

[0029] The wayland client uses the graphics library to draw the content to be displayed in the buffer to obtain the drawn buffer; the EGLsurface surface is created as the buffer through EGL, and then the drawing data is directly written to the surface through opengl es; the EGLSurface surface is the client surface;

[0030] Submit the drawn buffer to the wayland server through the wayland client using the wayland protocol, or call the eglswapbuffer interface through the wayland client to submit the drawn buffer to the wayland server;

[0031] Receive the drawn buffer from a single Wayland client through the Wayland server and synthesize the received buffer into a final frame;

[0032] The synthesized frame is sent to the display device for display through the wayland server; the display device is managed by the wl_output protocol.

[0033] Furthermore, in the WaylandShellSurface component, the specific method for creating a surface with the Wayland client is:

[0034] Initialize Wayland display: connect to Wayland display server;

[0035] Create wl_surface or EGLSurface: use the corresponding interface; wl_surface and EGLSurface are both the surfaces of the wayland client;

[0036] Set properties of the surface with the Wayland client: Set various properties for the surface with the Wayland client as needed, including buffers and input events for the Wayland client.

[0037] Furthermore, in the WaylandShellSurface component, the specific method for managing the surface with the Wayland client is:

[0038] Submit buffer: submit the drawing content to the wayland server;

[0039] Process input events: process input events forwarded by wayland client;

[0040] Destroy surface: When the corresponding surface is no longer needed, call the corresponding destruction function to destroy it.

[0041] Furthermore, in the WaylandShellSurface component, the state of the surface includes:

[0042] Attached state, submitted state, visible state, damaged state, focused state, and locked state.

[0043] Provided is a Wayland interaction method based on a system supporting Qt4 adaptation to Wayland, comprising the following steps:

[0044] S1. Initialize qapplication of qt4 and load QWayland module during the initialization process;

[0045] S2, using Qt4 native API to draw the developer's target interface through QtGui module;

[0046] S3, process the wayland event information through the QWayland module, and call the Qt event processing module to encapsulate the wayland client event;

[0047] S4. Distribute the events of the encapsulated wayland client to the event processing framework of qt4 to complete the adaptation of qt4 to wayland.

[0048] Furthermore, the specific method of processing the information of the wayland event through the QWayland module and calling the Qt event processing module to encapsulate the event of the wayland client includes the following sub-steps:

[0049] A1. Connect to the wayland server through the QWayland module and obtain the services registered on the wayland server;

[0050] A2. Get EGL-related extensions and information;

[0051] A3. Configure EGL property settings;

[0052] A4. Create an EGL context to provide an environment for graphics rendering;

[0053] A5. Create an EGL surface to provide a target or container for graphics rendering;

[0054] A6. Load the xdg-shell protocol for basic communication;

[0055] A7. Load the wl_seat protocol to handle input devices and their interaction with the user interface.

[0056] A8. Load the wl_output protocol to manage display output;

[0057] A9. Load wp_text_input_manager_v2 protocol to manage text input;

[0058] A10, determine whether the currently rendered image is 3D, if it is 3D, proceed to step A11; if the currently rendered image is 2D, use QPainter to render the image, and proceed to step A12;

[0059] A11. Bind the EGL context to the current thread and use OpenGL es to render the current window.

[0060] A12, by calling the eglswapbuffer interface, submit the drawn buffer to the wayland server and display it on the monitor;

[0061] A13, block and wait for the wayland server to return the event notification, so that the wayland client and the wayland server are synchronized;

[0062] A14. Call the Qt event processing module to encapsulate the wayland client's events as qt4 events and add them to the qt4 event processing queue, and complete the response and interaction to the wayland client's events through the qt4 event processing framework.

[0063] The beneficial effects of the present invention are as follows: the present invention provides a more complete Wayland protocol, allowing Qt4 to provide more interactive functions under Wayland, and improves the event processing mechanism for Wayland in Qt4, so that Qt4 can adapt to Wayland versions above 1.0, providing developers with more friendly development tools. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 This is the structural diagram of this system;

[0065] Figure 2 Schematic diagram of software layout in the embodiment. DETAILED DESCRIPTION

[0066] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0067] like Figure 1 As shown, the system supporting Qt4 adaptation to Wayland includes QtGui module, Qt event processing module and QWayland module; wherein:

[0068] QtGui module, used to draw the developer's target interface through the Qt4 native API;

[0069] Qt event processing module, which is used to encapsulate the events of the wayland client through the qpa plug-in and distribute the encapsulated wayland client events to the event processing framework of qt4;

[0070] The QWayland module is loaded into the Qt4 development framework as a plug-in to process the information of the Wayland event and call the Qt event processing module to encapsulate the events of the Wayland client.

[0071] In this embodiment, the QWayland module includes a QWaylanddisplay submodule, a QWaylandwindow submodule, a QWaylandInputDevice submodule, a QPlatformScreen submodule, and a QWaylandGLIntegration submodule; wherein:

[0072] The QWaylanddisplay submodule is used to establish and maintain a connection with the wayland server. When an event sent from the wayland server to the wayland client is blocked on the wayland client, it determines whether the current thread is processing an event. If so, the thread's task queue is immediately cleared and the event from the wayland server is processed immediately. Otherwise, the event from the wayland server is processed immediately. By maintaining a window array and a surface array, the wayland client with multiple windows and surfaces is managed. In the QWaylanddisplay submodule, the types of protocols are updated to wayland1.0 or above, including xdg-shell protocol, wl-shell protocol, wl-seat protocol, xdg-decoration protocol, and zwp_text_input_manager_v2 protocol.

[0073] There are two different rendering methods in Wayland, one is shm and the other is EGL, so two windows are defined in Qt. EGL is an interface used to connect the OpenGL es rendering API with the local window system. It is usually used for efficient graphics rendering, especially hardware-accelerated scenes. Due to the use of hardware acceleration, EGL usually provides better performance. As for SHM, it is a simple way to use shared memory, allowing multiple processes to access the same memory area, used to transfer image data from one process to another, usually used for simple graphics display. In SHM, the image buffer is created through shared memory, which is suitable for uncomplicated rendering scenes.

[0074] In this embodiment, the QWaylandwindow submodule is used to render and display the window of the Wayland client. In order to adapt to the shm and EGL of Wayland version 1.0 and above, the qwaylandwindow submodule improves the shm and EGL rendering steps and replaces them with the API of Wayland 1.0 and above as follows:

[0075] 1. Create and initialize shared memory: First create a shared memory object (buffer) through the wayland client, which usually uses the shm extension or EGL. The wayland client needs to specify the size and format of the buffer. Map the created shared memory to the client's address space so that the memory area can be directly accessed and modified.

[0076] 2. Rendering content: The wayland client draws the content to be displayed in shared memory. This can be achieved by using a graphics library (such as Cairo, OpenGL es, Qt's own QPainter graphics library, etc.), depending on the graphics API used. After the drawing is completed, the wayland client updates the content in the shared memory and prepares to send it to the wayland server.

[0077] 3. Submit buffer: The wayland client submits the drawn buffer to the wayland server through the wayland protocol, or uses the interface eglswapbuffer to submit the buffer to the wayland server through EGLSurface. This usually involves sending a wl_surface.commit request, or calling the EGLSwapBufer command to inform the wayland server that the new image content is ready. After receiving the submission request, the wayland server will process the buffer and add it to the synthesis queue.

[0078] 4. Synthesis and display: The wayland server receives buffers from a single wayland client and synthesizes them into a final frame. After the synthesis is completed, the synthesized frame is sent to the display device for display through the wayland server; the display device is managed by the wl_output protocol.

[0079] Since the two display graphics modes in Wayland are divided into two windows, QWaylandEGLWindow and QWaylandSHMWindow, the function of the QWaylandwindow submodule is mainly to manage these two windows and manage window rendering methods.

[0080] In the specific implementation process, the functions of the QWaylandInputDevice submodule are as follows:

[0081] 1. Input device management: The QWaylandInputDevice submodule is responsible for exchanging information with the Wayland server and managing the status and properties of the input device. During the initialization phase, the QWaylandInputDevice module sets the initial state and default properties, records them, and updates the state every time an interactive operation occurs.

[0082] 2. Event processing: Process events from input devices, such as key events, mouse movement and click events, and touch events, by encapsulating Wayland client input events as Qt4 framework events and forwarding them to the corresponding window.

[0083] 3. Status update: Monitor the connection status and type changes of input devices. When the connection status and type of input devices change, issue corresponding event types to ensure that the application can respond to changes in input devices in real time.

[0084] The QWaylandInputDevice submodule provides support for 0~9 numbers and a~z Latin characters, special keys and key combinations, and left and right key click events. Compared with qt4, the keyboard key types and special character support are improved.

[0085] In the specific implementation process, the functions of the QPlatformScreen submodule are as follows:

[0086] 1. Screen information: Provides basic information about the screen, such as size, resolution, pixel density (DPI), available area, etc.

[0087] 2. Multi-screen support: Supports applications running on multiple displays, allowing developers to obtain information about all connected screens and manage the display of windows on different screens.

[0088] 3. Screen properties: Provides screen properties, such as color depth, refresh rate, etc., to help developers optimize user interface and graphics rendering.

[0089] 4. System integration: Integrate with the operating system's window management system to ensure that the application can correctly handle screen changes (such as adding or removing monitors).

[0090] 5. Event processing: Able to receive and process screen-related events, such as changes in screen resolution.

[0091] In the specific implementation process, the functions of the QWaylandGLIntegration submodule are as follows:

[0092] 1. OpenGL es context management: responsible for creating and managing OpenGL es context for graphics rendering in the Wayland environment.

[0093] 2. Interaction with the Wayland server: Communicate with the Wayland server to handle the resources and status required in the OpenGL es rendering process.

[0094] 3. Frame buffer management: handles the creation, update, and destruction of frame buffers, which is very important for real-time rendering and display.

[0095] 4. Event processing: Receive events from windows and input devices, and perform corresponding processing in the OpenGL es rendering loop.

[0096] 5. Extension support: OpenGL es extensions are provided, allowing developers to take advantage of modern graphics features to enhance the performance of applications.

[0097] 6. Log support: Log support is provided. When it is found that the graphics are not rendered correctly, you can quickly know whether it is due to the current qt4 not supporting or the code problem.

[0098] In the specific implementation process, the QWaylandSurface component adds an interaction interface and a buffer management interface with the wayland server based on the wayland1.0 framework API, and modifies the interface for creating wl_surface or EGLSurface based on the wayland1.0 framework API; the functions of the QWaylandSurface component are as follows:

[0099] 1) Surface management: responsible for creating and managing surfaces with the Wayland client. These surfaces can be windows, icons or other visual elements. The specific creation method is as follows:

[0100] 1. Initialize Wayland display: first connect to the Wayland display server;

[0101] 2. Create wl_surface or EGLSurface: use the corresponding interface;

[0102] 3. Set the properties of the surface: Set various properties for the surface as needed, such as buffer and input events;

[0103] The specific surface methods of managing the Wayland client are as follows:

[0104] 1. Submit buffer: submit the drawing content to the surface;

[0105] 2. Process input events: process input events forwarded by the wayland server;

[0106] 3. Destroy the surface: When the surface is no longer needed, the corresponding destruction function can be called;

[0107] 2) Interaction with the Wayland server: Encapsulate the interaction interface in Wayland to provide interaction functions with the Wayland server and handle surface-related requests and events;

[0108] 3) Buffer management: Maintain the state of the surface. The maintained state is as follows:

[0109] Attachment state: This state indicates whether there is currently a buffer attached to the surface. When a buffer is attached, the surface can be prepared for rendering.

[0110] Commit status: indicates the status of the last operation on the surface. After calling the commit method, the content and status of the surface are submitted to the wayland client.

[0111] Visible state: simply whether the surface is visible. If the surface is occluded or not visible, this state affects its rendering updates.

[0112] Damage state: If an area of ​​the surface needs to be updated, it is marked as damaged and the damaged state triggers a repaint.

[0113] Focus state: Indicates whether the surface has input focus. During user interaction, the focus state is critical for receiving input events.

[0114] Locked state: In some cases, a surface may be locked to prevent modification or input. This is typically used for modal dialogs.

[0115] Maintain the above state in order to respond appropriately when the user interacts (for example, clicking on another window causes the focus to switch, a new window pops up, or a new window pops up):

[0116] 4) Input event processing: It can receive input events from the Wayland server, such as mouse clicks and keyboard input, and forward them to the corresponding window or component.

[0117] In the specific implementation process, the QWaylandBuffer component functions as follows:

[0118] 1. Buffer management: The QWaylandBuffer component is responsible for transferring graphics data between the application and the Wayland server. The data can be as follows:

[0119] Additional metadata: Data metadata related to the buffer can be transmitted, such as buffer format, size, coordinate information, etc. This information helps the receiving end to correctly parse and use the buffer.

[0120] Sync information data: In some cases, a buffer can contain sync information that indicates when it is safe to use or modify the buffer. This is particularly important in multithreaded or asynchronous rendering scenarios.

[0121] Color space information data: In order to support different display devices, it is sometimes necessary to transmit color space information to ensure color accuracy and consistency.

[0122] Format information data: includes the format of the buffer (such as RGB, YUV, etc.), which is important for decoding and correctly displaying the image.

[0123] User-defined data: Developers can add additional user data in custom buffer views to meet specific needs.

[0124] 2. Association with the surface: The buffer (i.e. shared memory) is usually associated with one or more QWaylandSurface components. Since the data of a buffer can be used on multiple surfaces, it is a one-to-many relationship. The QWaylandBuffer component allows the graphics content to be associated with a specific surface.

[0125] 3. Multiple format support: Supports multiple image formats, such as RGBA, RGB, etc., which can be configured as needed through the existing interface of the waylad framework.

[0126] 4. Performance optimization: Because the QWaylandBuffer component uses a shared memory mechanism, the data in it can be directly accessed by the CPU. The QWaylandBuffer component can achieve efficient data transmission, reduce CPU overhead, and improve rendering performance.

[0127] 5. Lifecycle management: handles the lifecycle of the buffer, including operations such as creation, update and destruction. Creation operation: usually occurs when an image needs to be drawn; update operation: occurs when the data inside the buffer changes; destruction operation: usually occurs after the data is prepared on the client and sent to the wayland server, to ensure effective management of resources.

[0128] In the specific implementation process, the QWaylandShellSurface component adds an interaction interface and a buffer management interface with the wayland server based on the wayland1.0 framework API, and modifies the interface for creating wl_surface or EGLSurface based on the wayland1.0 framework API. Its functions are as follows:

[0129] 1. Shell surface management: responsible for creating and managing the surface that interacts with the Wayland shell, including the window's life cycle, position, size and other properties.

[0130] 2. User interface elements: Allow developers to create traditional desktop windows, such as resizable windows, full-screen windows, etc.

[0131] 3. State management: handle different states of the window, such as maximized, minimized, and normal states, and be able to respond to user input events.

[0132] 4. Interaction support: integrated input event processing, able to receive mouse and keyboard events from users and perform corresponding processing.

[0133] 5. Communication with the Wayland server: By communicating with the Wayland server, the window position and size are updated to ensure that the window meets the user's expectations when displayed. This is determined according to the strategy of the Wayland server.

[0134] In the QWaylandShellSurface component, the management of the top-level window is provided; it is encapsulated through the interface of Wayland1.0, and a new interface for communicating with the Wayland server is added to ensure that the Wayland server and the Wayland client in the Wayland framework can maintain data consistency. The decoration option refers to the toolbar at the top of the window, with the zoom in, zoom out, and close buttons on the far right. The decoration option is managed by the Wayland server and uses the xdg-decoration protocol provided by the Wayland framework. The QWaylandShellSurface component also provides an interface for setting some properties of the top-level window, such as name, size, and other properties.

[0135] The QWaylandShellIntegration component loads the xdg-shell protocol and wl-shell protocol as plug-ins based on the QWaylandShellintegration module of qt4 to create corresponding windows and surfaces.

[0136] In the specific implementation process, since the EGL + opengl es solution has higher performance, can support hardware acceleration, and has scalable configuration and supports more complex rendering scenarios, the entire qtwayland module uses the EGL + opengles rendering method by default.

[0137] In this embodiment, the Wayland interaction method based on the system supporting Qt4 adaptation to Wayland includes the following steps:

[0138] S1. Initialize qapplication of qt4 and load QWayland module during the initialization process;

[0139] S2, using Qt4 native API to draw the developer's target interface through QtGui module;

[0140] S3, process the wayland event information through the QWayland module, and call the Qt event processing module to encapsulate the wayland client event;

[0141] S4. Distribute the events of the encapsulated wayland client to the event processing framework of qt4 to complete the adaptation of qt4 to wayland.

[0142] In the specific implementation process, the specific method of processing the information of the wayland event through the QWayland module and calling the Qt event processing module to encapsulate the event of the wayland client includes the following sub-steps:

[0143] A1. Connect to the wayland server through the QWayland module and obtain the services registered (that is, supported) on the wayland server;

[0144] A2. Get EGL related extensions and information;

[0145] Because qt4 supports an earlier version of OpenGL es, extended support is needed to support different functions. Examples of extensions are as follows:

[0146] 1. EGL_KHR_image_base: supports the creation and use of image objects for rendering in OpenGL es.

[0147] 2. EGL_KHR_image_pixmap: Allows creation of pixmap surfaces in EGL that can be shared with images, thereby improving performance and compatibility.

[0148] 3. EGL_EXT_platform_base: Provides platform-independent functionality so that EGL can run on multiple platforms, including Android and Linux.

[0149] 4. EGL_EXT_platform_device: Provides access to the device and supports sharing contexts between different types of devices (such as GPU and CPU).

[0150] 5. EGL_EXT_create_context_robustness: Enhances the robustness of the context to prevent application crashes due to errors.

[0151] 6. EGL_KHR_context_flush_control: Provides control over context refresh, allowing developers to optimize rendering performance.

[0152] EGL_KHR_swap_buffers_with_damage: Supports specifying damaged areas when swapping buffers, thereby optimizing the efficiency of redrawing.

[0153] A3. Configure EGL property settings;

[0154] The main purpose of configuring EGL attributes is to specify the desired characteristics and behaviors when creating contexts and surfaces. These attributes affect the performance, quality, and compatibility of graphics rendering. The following are some common EGL attributes and their purpose:

[0155] 1. EGL_RENDERABLE_TYPE: Specifies the type of rendering that can be used, such as OpenGL ES or OpenGL. Make sure that the context created supports the required graphics API.

[0156] 2. EGL_SURFACE_TYPE: Defines the type of surface that can be created, such as window surface, pixmap surface, or off-screen surface, to ensure compatibility with the target display system.

[0157] 3. EGL_CONFIG_ID: Used to select a specific configuration to ensure that the application's requirements for color depth, number of buffers, etc. are met.

[0158] 4. EGL_BUFFER_SIZE: Specifies the size of the color buffer (in bits), which affects the color performance of the image.

[0159] 5. EGL_DEPTH_SIZE: Sets the size of the depth buffer, which is used to process depth information in three-dimensional scenes and improve rendering accuracy.

[0160] 6. EGL_STENCIL_SIZE: Defines the size of the template buffer, which is usually used for complex graphics operations and effects.

[0161] 7. EGL_ALPHA_SIZE: Specifies the size of the Alpha channel, affecting transparency processing and blending effects.

[0162] 8. EGL_SAMPLE_BUFFERS and EGL_SAMPLES: Control the enabling and number of samples of multi-sampling anti-aliasing, improve graphics quality and reduce aliasing.

[0163] 9. EGL_SWAP_BEHAVIOR: Defines buffer swap behavior, such as whether to use the default or separate swap mode, affecting the effect of frame rendering.

[0164] A4. Create an EGL context to provide an environment for graphics rendering, allowing the application to interact with the underlying graphics API, and create an EGL context using the selected configuration and required properties;

[0165] A5. Create an EGL surface to provide a target or container for graphics rendering so that the rendering results can be displayed on the screen or used for subsequent image processing;

[0166] A6. Load the xdg-shell protocol for basic communication;

[0167] xdg-shell is a protocol for wayland, and its main functions are as follows:

[0168] 1. Window management: allows the client to create, configure and manage its windows so that the windows can be displayed correctly on the screen.

[0169] 2. State management: support various window states, such as maximize, minimize and full screen.

[0170] 3. Positioning and size adjustment: Provide an interface to set the position and size of the window and support responding to user operations.

[0171] 4. Server support: Combined with the Wayland server, it can handle the display and rendering of windows.

[0172] 5. Extensibility: Allow other protocols or extensions to be integrated with xdg-shell to provide more functionality and flexibility.

[0173] A7. Load the wl_seat protocol to handle input devices and their interaction with the user interface. The specific functions are as follows:

[0174] 1. Input device management: wl_seat allows clients to register and manage multiple devices, providing a unified interface to obtain input events.

[0175] 2. Event distribution: It distributes events from input devices (such as key presses, movements, clicks) to the appropriate windows or applications.

[0176] 3. Focus management: supports the management of input focus and determines which window currently accepts input events.

[0177] A8. Load the wl_output protocol to manage display output. The specific functions are as follows:

[0178] 1. Display Information: Provides basic information about the output device, such as device name, physical size, resolution, and current mode.

[0179] 2. Screen geometry: Ability to obtain aggregate information of output devices, including position and size, to help clients arrange windows and content correctly.

[0180] 3. Mode support: Supports multiple display modes, allowing the client to query available modes and select appropriate resolution and refresh rate.

[0181] 4. Status notification: When the status of the current device changes, the wl_output protocol will send the corresponding time notification to the client.

[0182] 5. Multi-monitor support: Allows the system to support multiple output devices so that applications can handle multi-screen environments.

[0183] A9. Load the wp_text_input_manager_v2 protocol to manage text input. The main purpose is to provide a standardized way for applications to manage and process text input, including input method support and cursor control. The specific functions are as follows:

[0184] 1. Text input management: Provides an interface to register and manage text input, allowing applications to receive text input from users.

[0185] 2. Input method support: Integrated input method framework allows users to input text through different input methods (such as pinyin, strokes, etc.).

[0186] 3. Cursor management: Allows the client to control the position and style of the text cursor to provide visual feedback during text input;

[0187] A10, determine whether the currently rendered image is 3D, if it is 3D, proceed to step A11; if the currently rendered image is 2D, use QPainter to render the image, and proceed to step A12;

[0188] A11. Bind the EGL context to the current thread and use OpenGL es to render the current window.

[0189] A12, by calling the eglswapbuffer interface, submit the drawn buffer to the wayland server and display it on the monitor;

[0190] A13, block and wait for the wayland server to return the event notification, so that the wayland client and the wayland server are synchronized;

[0191] A14. Call the Qt event processing module to encapsulate the wayland client's events into qt4 events and add them to the qt4 event processing queue, and complete the response and interaction to the wayland client's events through the qt4 event processing framework.

[0192] The specific purposes and blocking times are as follows:

[0193] 1. Event-driven: Wayland uses an event-driven model. The client will not be blocked all the time, but will process multiple events in the event loop, including input events, graphics updates, etc.

[0194] 2. Avoid race conditions: Wayland is an event-driven system, and blocking waits can help prevent race conditions caused by multiple threads or processes accessing resources at the same time. This synchronization method can improve the stability of the program.

[0195] 3. Improve responsiveness: By receiving notifications from the Wayland server in a timely manner, the client can respond to user input or other events more quickly, providing a smoother user experience.

[0196] 4. Resource management: The Wayland server is responsible for managing display content and resource allocation. Waiting for its notification can ensure that the client operates at the right time and avoid resource conflicts or unnecessary requests.

[0197] 5. Collaboration and coordination: In a multi-client environment, blocking and waiting for notifications from the Wayland server can ensure smoother collaboration between clients and avoid problems caused by state asynchrony.

[0198] Blocking time problem:

[0199] 1. Event-driven: Wayland uses an event-driven model. The Wayland client will not be blocked all the time, but will process multiple events in the event loop, including input events, graphics updates, etc.

[0200] 2. Non-blocking design: When the Wayland client is waiting for notification from the Wayland server, it usually does not block completely, but stays in the event loop to handle other tasks, thus avoiding the situation where the application becomes unresponsive.

[0201] 3. Short-term blocking: The blocking time depends on the response speed of the Wayland server and the current system load, usually in milliseconds, but the specific time varies from situation to situation.

[0202] 4. Timeout handling: Some implementations may set a timeout mechanism to prevent problems caused by long waiting. If no notification is received within the set time, the Wayland client can choose to retry or perform error handling.

[0203] A13. Call the Qt event processing module to encapsulate the wayland client's events as qt4 events and add them to the qt4 event processing queue, and complete the response and interaction to the wayland client's events through the qt4 event processing framework.

[0204] For example, when a mouse event occurs, the QWayland module will encapsulate the mouse event as a mouse event in Qt and add it to the Qt event processing queue. This queue will allow the correct window to obtain the event and respond to it correctly. When a keyboard event occurs, the QWayland module will encapsulate the keyboard event as a keyboard event in Qt and add it to the Qt event processing queue. This queue will allow the correct window to obtain the event and respond to it correctly.

[0205] In one embodiment of the present invention, the entire software layout is as follows Figure 2 As shown, there are four buttons on the upper left and four windows below. By default, nothing is displayed in the four windows. When the button with the corresponding number is pressed, the window with the corresponding number is displayed. In the initialization process of the software program, we need to use the qapplication and qmainwindow modules as the canvas of the entire software. The four buttons and four windows in the figure above are drawn on the canvas. First, when initializing qapplication, the required surface data will be obtained through the QWayland module, and then the following controls will be drawn on the surface. The process is as follows:

[0206] 1. The function of window 1 is to display a cube, using the qglwidget module. The qglwidget module is a control for 3D display. It uses the module of the original framework of qt4. The entire display process is as follows:

[0207] ①. Because the wayland interaction process has been passed, the surface and OpenGL es context are created.

[0208] ②. Load custom vertex data and color data into the program. Vertex data is used to confirm the position and shape of graphics when drawing 3D graphics. Color data is used to determine the color and transparency of the image to be drawn. Then the defined data is stored in the buffer.

[0209] ③. Call the EGLSwapBuffer interface to exchange the buffer and render the 3D graphics to the window, making the cube visible on the client.

[0210] 2. Window 2 is an input text control, which uses the qtextedit module. The function of this module is to allow users to enter text in the window. It uses the module of the original framework of qt4. The entire display process is as follows:

[0211] ①. Because the surface has been created through the wayland interaction process, and because it is a 2D graphic, the qpianter control in qt4 is called afterwards to perform 2D drawing. After the drawing is completed, eglswapbuffer is called to make the entire qtextedit control visible on the client.

[0212] ②. When the user clicks on the qtextedit control window, first, the QWaylanddisplay submodule will obtain the notification from the wayland server, and the QWaylanddisplay submodule will send the notification to the QWaylandInputDevice submodule to receive the corresponding notification event processing interface, which is the wl_pointer subprotocol in wl_seat in the QWaylandInputDevice submodule. Then, the clicked mouse button (left or right), the click position and other attributes are encapsulated as a mouse click event in qt4 and sent to the corresponding qtextedit control. Here, the left mouse button click event is sent.

[0213] ③. The QWaylandSurface component will handle the focus status and object changes, that is, change the focus control to the current target control.

[0214] ④. When 0~9 or a~z is input from the keyboard, the QWaylanddisplay submodule will first obtain the notification from the wayland server, and the QWaylanddisplay submodule will send the notification to the QWaylandInputDevice submodule to receive the corresponding notification event processing interface, which is the wl_keyboard subprotocol in the wl_seat protocol in the QWaylandInputDevice submodule. Then the input keyboard button and the Latin character representation set defined in qt4 will be converted, and then encapsulated as a keyboard press event in qt4 and sent to the corresponding qtextedit control. Here, it is assumed that the keyboard event of the letter w is sent.

[0215] ⑤. When the qtextedit control receives a keyboard event, it will first determine the type of key. Here, the ordinary key w is transmitted, and the character drawing module is called to draw the w character on the EGLSurface (buffer).

[0216] 3. Window 3 is composed of three progress bars. Using the qprogressbar control in qt4, the specific interactive process is as follows:

[0217] ①, Because the surface has been created through the wayland interaction process.

[0218] ②. Because it is a 2D graphic, the painter control in qt4 is called afterwards, and eglswapbuffer is called to make the control visible on the client.

[0219] ③. When the user clicks the qprogressbar control, the QWaylanddisplay submodule will receive the notification from the wayland server, and the QWaylanddisplay submodule will send the notification to the event processing interface in the QWaylandInputDevice submodule to receive the corresponding notification, which is the wl_pointer subprotocol in the wl_seat protocol in the QWaylandInputDevice submodule. Here, the click position and the click button type (left or right mouse button) will be encapsulated as a qt4 mouse click event and sent to the qprogressBar control.

[0220] ④. In the QWaylandSurface component, the focus status and object changes will be processed, that is, the focus control will be switched to the current target control.

[0221] ⑤. After receiving the mouse click event, the qporgressBar control will determine the type of mouse click, which is the left button here, and then determine whether to set the progress bar progress through the type, call qpainter to draw the progress bar, and finally call eglSwapBuffer to make the graphics visible on the client.

[0222] 4. Window 4 is composed of nine qwidget controls in qt4. The specific display process is similar to the above and will not be repeated.

[0223] In summary, the present invention provides a more complete Wayland protocol, allowing Qt4 to provide more interactive functions under Wayland, and improves the event processing mechanism for Wayland in Qt4, so that Qt4 can adapt to Wayland versions above 1.0, providing developers with more friendly development tools.

Claims

1. A system that supports Qt4 adaptation to Wayland, characterized in that: Including QtGui module, Qt event processing module and QWayland module; among them: QtGui module, used to draw the developer's target interface through the Qt4 native API; Qt event processing module, which is used to encapsulate the events of the wayland client through the qpa plug-in and distribute the encapsulated wayland client events to the event processing framework of qt4; The QWayland module is loaded into the Qt4 development framework as a plug-in to process the information of the Wayland event and call the Qt event processing module to encapsulate the events of the Wayland client; The specific method in which the QWayland module processes the information of the wayland event and calls the Qt event processing module to encapsulate the event of the wayland client includes the following sub-steps: A1. Connect to the wayland server through the QWayland module and obtain the services registered on the wayland server; A2. Get EGL related extensions and information; A3. Configure EGL property settings; A4. Create an EGL context to provide an environment for graphics rendering; A5. Create an EGL surface to provide a target or container for graphics rendering; A6. Load the xdg-shell protocol for basic communication; A7. Load the wl_seat protocol to handle input devices and their interaction with the user interface. A8. Load the wl_output protocol to manage display output; A9. Load wp_text_input_manager_v2 protocol to manage text input; A10, determine whether the currently rendered image is 3D, if it is 3D, proceed to step A11; if the currently rendered image is 2D, use QPainter to render the image, and proceed to step A12; A11. Bind the EGL context to the current thread and use OpenGL es to render the current window. A12, by calling the eglswapbuffer interface, submit the drawn buffer to the wayland server and display it on the monitor; A13, block and wait for the wayland server to return the event notification, so that the wayland client and the wayland server are synchronized; A14. Call the Qt event processing module to encapsulate the wayland client's events as qt4 events and add them to the qt4 event processing queue, and complete the response and interaction to the wayland client's events through the qt4 event processing framework.

2. A system supporting Qt4 adaptation to Wayland according to claim 1, characterized in that: The QWayland module includes the QWaylanddisplay submodule, QWaylandwindow submodule, QWaylandInputDevice submodule, QPlatformScreen submodule, and QWaylandGLIntegration submodule; among them: The QWaylanddisplay submodule is used to establish and maintain the connection with the Wayland server. When an event sent from the Wayland server to the Wayland client is blocked on the Wayland client, it determines whether the current thread is processing an event. If so, the thread's task queue is immediately cleared and the event from the Wayland server is processed immediately. Otherwise, the event from the Wayland server is processed immediately. A window array and a surface array are maintained to manage the Wayland client with multiple windows and surfaces. QWaylandwindow submodule, used to render and display the wayland client window; The QWaylandInputDevice submodule is used to manage input devices, process events from input devices, call the Qt event processing module to encapsulate the wayland client input events as qt4 events, and forward them to the corresponding window; monitor the connection status and type changes of the input device, and issue the corresponding event type when the connection status and type of the input device change, so as to ensure that the application can respond to the changes of the input device in real time; The QPlatformScreen submodule is integrated with the window management system of the operating system to provide screen information, screen attributes, support multi-screen display, and receive and process screen-related events; The QWaylandGLIntegration submodule is used to manage the OpenGL es context; interact with the wayland server to handle the resources and status required in the OpenGL es rendering process; manage the frame buffer; receive events from windows and input devices and perform corresponding processing in the OpenGL es rendering loop; provide OpenGL es extensions; and provide logging support.

3. A system supporting Qt4 adaptation to Wayland according to claim 2, characterized in that: The QWaylandwindow submodule includes the QWaylandSurface component, the QWaylandBuffer component, the QWaylandShellSurface component, and the QWaylandShellIntegration component; among them: QWaylandSurface component, based on the wayland1.0 framework api, adds the interaction interface and buffer management interface with the wayland server, and modifies the interface for creating wl_surface or EGLSurface based on the wayland1.0 framework api; used to create and manage surfaces with wayland clients, encapsulate the interaction interface in wayland, provide interaction functions with the wayland server, and handle surface-related requests and events; maintain the state of the surface in the buffer, receive input events from the wayland server, and forward them to the corresponding window or component; QWaylandBuffer component, by adding buffer and Wayland server notification mechanism, sets status symbols for buffer, used to transfer graphics data between application and Wayland server; performs surface association, associates graphics content to a specific surface; performs format support configuration; performs performance optimization through shared memory mechanism; handles the life cycle of buffer; WaylandShellSurface component, used to create and manage surfaces that interact with the Wayland shell; manage user interface elements and provide support for developers to create traditional desktop windows; handle different states of windows, respond to, receive and process events from the Wayland client; provide decoration options, implement communication with the Wayland server by encapsulating the Wayland1.0 interface, and notify the decoration options through the server protocol to update the window position and size to ensure that the window meets the user's expectations when displayed; The QWaylandShellIntegration component loads the xdg-shell protocol and wl-shell protocol as plug-ins based on the QWaylandShellintegration module of qt4 to create corresponding windows and surfaces.

4. A system supporting Qt4 adaptation to Wayland according to claim 3, characterized in that: In the QWaylandBuffer component, the graphics data passed includes data metadata related to the buffer, synchronization information data used to indicate when the buffer can be safely used or modified, color space information data, buffer format data, and additional user data added by the developer in the custom buffer view.

5. The system supporting Qt4 adaptation to Wayland according to claim 3, characterized in that: In the QWaylandwindow submodule, the specific rendering method is: Use EGL to create at least one buffer through the wayland client, specify the size and format of the buffer, and map the created buffer to the address space of the wayland client; The wayland client uses the graphics library to draw the content to be displayed in the buffer to obtain the drawn buffer; the EGLsurface surface is created as the buffer through EGL, and then the drawing data is directly written to the surface through opengl es; the EGLSurface surface is the client surface; Submit the drawn buffer to the wayland server through the wayland client using the wayland protocol, or call the eglswapbuffer interface through the wayland client to submit the drawn buffer to the wayland server; Receive the drawn buffer from a single Wayland client through the Wayland server and synthesize the received buffer into a final frame; The synthesized frame is sent to the display device for display through the wayland server; The display device is managed by the wl_output protocol.

6. The system supporting Qt4 adaptation to Wayland according to claim 3, characterized in that: In the WaylandShellSurface component, the specific method for creating a surface with the Wayland client is: Initialize Wayland display: connect to Wayland display server; Create wl_surface or EGLSurface: use the corresponding interface; wl_surface and EGLSurface are both the surfaces of the wayland client; Set properties of the surface with the Wayland client: Set various properties for the surface with the Wayland client as needed, including buffers and input events for the Wayland client.

7. The system supporting Qt4 adaptation to Wayland according to claim 3, characterized in that: In the WaylandShellSurface component, the specific method for managing the surface with the Wayland client is: Submit buffer: submit the drawing content to the wayland server; Process input events: process input events forwarded by wayland client; Destroy surface: When the corresponding surface is no longer needed, call the corresponding destruction function to destroy it.

8. The system supporting Qt4 adaptation to Wayland according to claim 3, characterized in that: In the WaylandShellSurface component, the state of the surface includes: Attached state, submitted state, visible state, damaged state, focused state, and locked state.

9. A Wayland interaction method for a system supporting Qt4 adaptation to Wayland based on any one of claims 1 to 8, characterized in that: The following steps are involved: S1. Initialize qapplication of qt4 and load QWayland module during the initialization process; S2, using Qt4 native API to draw the developer's target interface through QtGui module; S3, process the wayland event information through the QWayland module, and call the Qt event processing module to encapsulate the wayland client event; S4. Distribute the events of the encapsulated wayland client to the event processing framework of qt4 to complete the adaptation of qt4 to wayland.

Images