Method and system for dragging x application data under wayland environment

Abstract

The application discloses a method and system for dragging X application data in a wayland environment, and the method is applied to a wayland compositor and comprises the following steps: waiting for and acquiring declaration information of an X11 protocol of a drag source window, and creating a corresponding X11 application window; judging an application type of a drag target window, and if the application type of the drag target window is a Wayland application, using the created X11 application window to accept an event of an Xdnd protocol of the drag source window and X11 application data of the drag source window; converting the event message of the Xdnd protocol into a wayland drag protocol, and then sending the event message of the wayland drag protocol and the X11 application data of the drag source window to the drag target window. The application solves the problem that data in an X11 application cannot be dragged into a Wayland application in a Wayland environment.

Description

Technical Field

[0001] This invention relates to the field of computer graphics display technology, specifically to a method and system for dragging and dropping X application data in a Wayland environment. Background Technology

[0002] Wayland is a protocol that defines how Wayland clients communicate with Wayland compositors. Given Wayland's advantages over X11, particularly its better support for high resolutions and high refresh rates, the open-source community has fully switched to Wayland for development, and major system distributions have begun using Wayland as the default graphics display protocol. In desktop systems, drag and drop allows users to drag one or more files from the source interface to the target interface using the mouse, providing visual feedback during the dragging process.

[0003] In the X11 environment, dragging and dropping data is implemented through the Xdnd protocol. The flowchart of the protocol and data transfer is as follows: Figure 1 As shown, the drag-and-drop source declares ownership via the XdndSelection message, and then interacts with the target using the XdndEnter, XdndStatus, XdndPosition, XdndDrop, and XdndFinished messages. The drag-and-drop target requests data via the XdndSelection message, similar to a copy-and-paste process, and the drag-and-drop source writes the data to the target's window properties.

[0004] In the Wayland environment, data is uniformly managed by the synthesizer, resulting in greater efficiency and a more modern approach. Drag and drop data is implemented through the Wayland drag-and-drop protocol, which is part of the Wayland protocol suite. Unlike the X11 Xdnd protocol, the Wayland drag-and-drop protocol is more modern and efficient. The protocol transmission flowchart is as follows: Figure 2 As shown, it relies on interfaces such as wl_data_device (representing a data device used to manage drag-and-drop and copy-and-paste operations), wl_data_source (representing a data source, created by the drag-and-drop source client), and wl_data_offer (representing a data provider object, received by the drag-and-drop target client). Furthermore, all these protocols are uniformly processed by the Wayland synthesizer.

[0005] Wayland and X11 use completely different communication protocols. The native Wayland application manages data exchange through the zwlr_data_control protocol, while the X application relies on the Xdnd protocol, and the protocol is passed between applications. The compositor cannot receive protocol events sent by the X application. Therefore, the existing compositor does not establish a conversion bridge between the two protocols, resulting in drag events being unable to be passed across protocols.

[0006] Chinese invention patent application "Screenshotting Method, System, and Medium for X Applications in a Wayland Environment" (CN202211240560.5) provides a screenshotting method, system, and medium for X applications in a Wayland environment. The method includes: intercepting a call request initiated by an X11 application to a specified screenshot interface via Xwayland; determining via Xwayland whether the call request is for retrieving root window content; if so, obtaining the current screen image from the Wayland compositor via Xwayland and returning it to the X11 application. This solves the problem that applications that retrieve window or screen content via the X11 protocol cannot use the Wayland environment. It does not involve the implementation of dragging and dropping X application data into a Wayland application within the Wayland environment.

[0007] Chinese invention patent application "A Method for Obtaining a Hardware Cursor in a Remote Desktop in a Wayland Environment" (CN202310823579.0) provides a method for obtaining a hardware cursor in a remote desktop in a Wayland environment, including the following steps: Step S1: Obtain the Wayland protocol for screen display data, adding two events, cursor-type and cursor-info; Step S2: The Wayland compositor obtains the system cursor type and sends the cursor-type event to the remote desktop server; Step S3: The Wayland compositor determines the cursor type; if it is a hardware cursor, it sends a cursor-info event to the remote desktop server; Step S4: The remote desktop server, based on the received cursor-info event, obtains the cursor's style and position data, destroys the previous cursor, and draws a new cursor. This does not involve the implementation of dragging and dropping X application data to a Wayland application in a Wayland environment. Summary of the Invention

[0008] The technical problem to be solved by the present invention is to provide a method and system for dragging and dropping X application data in the Wayland environment, which supports users in the Wayland environment to drag and drop data from the X11 application to the Wayland application.

[0009] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A method for dragging and dropping X application data in the Wayland environment, applied to the Wayland synthesizer, the method includes the following steps: Wait for and obtain the X11 protocol declaration information of the dragged source window, and create the corresponding X11 application window; Determine the application type of the target window to be dragged. If the application type of the target window is a Wayland application, use the created X11 application window to receive the Xdnd protocol event messages and X11 application data of the source window to be dragged. The event messages of the Xdnd protocol are converted to the Wayland drag protocol, and then the event messages of the Wayland drag protocol, along with the X11 application data of the drag source window, are sent to the drag target window.

[0010] Furthermore, when creating the corresponding X11 application window, specifically, an X11 application window is created that only has Xdnd protocol sending and receiving functions and does not have a specific rendering surface and buffer, in order to avoid the created X11 application window being displayed on the screen.

[0011] Furthermore, after creating the corresponding X11 application window, the process includes: mapping the created X11 application window onto the screen, using the created X11 application to accept the drag-and-drop entry message XdndEnter of the Xdnd protocol of the drag source window to obtain the drag data type, and then unmapping the created X11 application window from the screen.

[0012] Furthermore, when determining the application type of the dragged target window, the system specifically listens for mouse and touchscreen movement. If the mouse or touchscreen enters the window surface of a Wayland application, the application type of the dragged target window is a Wayland application; if the mouse or touchscreen enters the window surface of an X11 application, the application type of the dragged target window is an X11 application.

[0013] Furthermore, after determining the application type of the target window to be dragged, the process also includes the following steps: if the application type of the target window to be dragged is an X11 application, then the application type of the target window to be dragged is a Wayland application.

[0014] Furthermore, after sending the Wayland drag-and-drop protocol and the X11 application data of the drag-and-drop source window to the drag-and-drop target window, the process also includes: obtaining the drag-and-drop response message of the Wayland protocol during dragging, converting the drag-and-drop response message of the Wayland protocol into the Xdnd protocol, and sending it to the X11 application of the drag-and-drop source window through the created X11 application window.

[0015] This invention also proposes a system for dragging and dropping X application data in a Wayland environment, including a drag-and-drop source window and a target window. The application of the drag-and-drop source window is an X11 application, and the application of the target window is either an X11 application or a Wayland application. The system also includes a Wayland synthesizer. The Wayland synthesizer waits for and obtains the X11 protocol declaration information of the drag-and-drop source window, creates a corresponding X11 application window, and then determines the application type of the drag-and-drop target window. If the application type of the drag-and-drop target window is a Wayland application, the created X11 application window is used to receive the Xdnd protocol events and X11 application data of the drag-and-drop source window. Finally, the Xdnd protocol event messages are converted into the Wayland drag-and-drop protocol, and then the Wayland drag-and-drop protocol event messages and the X11 application data of the drag-and-drop source window are sent to the drag-and-drop target window.

[0016] The present invention also proposes an apparatus for dragging and dropping X application data in a Wayland environment, comprising a processor and a storage medium, wherein the storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the method for dragging and dropping X application data in a Wayland environment.

[0017] The present invention also proposes a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the steps of the method for dragging and dropping X application data in the Wayland environment.

[0018] The present invention also proposes a computer program product, comprising a computer program that is executed by a processor to implement the steps of the method for dragging and dropping X application data in the Wayland environment.

[0019] Compared with the prior art, the advantages of the present invention are as follows: This invention solves the problem of not being able to drag and drop data from an X11 application to a Wayland application in the Wayland environment by creating an X11 application in the Wayland synthesizer, enabling the Wayland synthesizer to receive drag and drop events and data from the X11 application, and then sending the data to the Wayland application. Attached Figure Description

[0020] Figure 1This is a flowchart illustrating the protocol and data transfer during data dragging and dropping in X11 applications within the X11 environment.

[0021] Figure 2 This is a flowchart illustrating the protocol and data transfer during drag-and-drop operations in the Wayland application within the Wayland environment.

[0022] Figure 3 This is a flowchart illustrating the process of dragging and dropping X11 application data into a Wayland application within the Wayland environment, as described in this embodiment of the invention. Detailed Implementation

[0023] The present invention will be further described below with reference to the accompanying drawings and specific preferred embodiments, but this does not limit the scope of protection of the present invention.

[0024] Before introducing specific embodiments of the present invention, the relevant concepts or terms will be explained.

[0025] Wayland: Wayland is a communication protocol and library for building display server and client applications. It is designed to replace the X Window System (X11) as the graphics display system on Linux and other operating systems.

[0026] The Wayland protocol defines the communication specifications between the server and client, including window management, input events, and buffer exchange. It uses the concepts of objects and interfaces to describe different functionalities and interacts through message passing. Several key concepts in the Wayland protocol are as follows: Client: A regular program that needs to draw on the screen.

[0027] Server (compositor): The core program of the Wayland protocol, responsible for combining the drawings from multiple clients and outputting them to the display.

[0028] Request: Wayland communication initiated by the client.

[0029] Event: An asynchronous notification initiated by the server and selectively received by the client; sometimes referred to as "notification".

[0030] XWayland: XWayland acts as both an X11 server and a Wayland client. XWayland's role is to convert the X11 protocol to the Wayland protocol or vice versa, allowing older X11 applications to work with Wayland-based display servers.

[0031] X11 applications: Various applications developed based on the X11 protocol that run on the X Window system.

[0032] The X11 protocol, also known as the X Window System, is a bitmap-based windowing system widely used in UNIX and UNIX-like operating systems to implement graphical user interfaces. The core of the X11 protocol consists of an X server and an X client. The X server manages display-related hardware devices such as monitors, graphics cards, and mice, while the X client handles application logic.

[0033] Xdnd Protocol: A protocol used in the X Window System to enable drag-and-drop operations across applications. It defines how applications negotiate data transfer, such as dragging and dropping files, text, or images.

[0034] The Wayland drag-and-drop protocol consists of interfaces such as wl_data_device (representing the data device used to manage drag-and-drop and copy-and-paste operations), wl_data_source (representing the data source, created by the drag-and-drop source client), and wl_data_offer (representing the data provider object, received by the drag-and-drop target client). `map`: Maps the window onto the screen. The window must be mapped before it can be seen by the user.

[0035] `unmap`: Unmaps the window from the screen. The window resources still exist, but are no longer visible; they can be restored to display later by remapping.

[0036] Example In the Wayland environment, when a user drags and drops data from an X11 application into a Wayland application, the drag-and-drop operation fails because neither the Wayland synthesizer nor the Wayland application can receive the Xdnd protocol data sent by the X11 application.

[0037] To address the aforementioned issues, this embodiment proposes a method for dragging and dropping X application data within the Wayland environment, allowing users to drag and drop data from X11 applications into Wayland applications within the Wayland environment. The approach is as follows: 1. When drag-and-drop begins, an X11 application window the size of the screen is created to receive the drag-and-drop entry message (XdndEnter) and save the supported data types associated with this drag-and-drop operation. Once complete, the X11 application window is created and then hidden from the screen using the unmap operation.

[0038] 2. When dragging and dropping into other applications, determine the type of the application. If it is an X11 application, no action is taken, and applications will interact normally through the Xdnd protocol.

[0039] 3. If it is a Wayland application, the previously created X11 application window will be displayed on the screen through a map operation to receive event messages such as drag-and-drop position message XdndPosition and drag-and-drop drop message XdndDrop in the Xdnd protocol.

[0040] 4. Convert the received Xdnd protocol into the corresponding Wayland protocol and send it to the Wayland application.

[0041] like Figure 3 As shown, the method of this embodiment is applied to a Wayland synthesizer and includes the following steps: S1) Wait for and obtain the X11 protocol declaration information of the dragged source window, and create the corresponding X11 application window; In this embodiment, the window of the X11 application whose data is being dragged serves as the drag source window. When data is dragged from the X11 application, the Wayland compositor receives the selection notify declaration information in the X11 protocol and creates an X11 application window the size of the screen to receive and send subsequent Xdnd protocol messages. To prevent the created X11 application window from obscuring the screen and affecting normal operation, the Wayland compositor code uses the window_id of the created X11 application window to determine whether to ignore the rendering, compositing, and input processing of the created X11 application window. That is, an X11 application window is created that only has Xdnd protocol sending and receiving functions, does not have a specific rendering surface or buffer, and does not affect normal screen rendering, compositing, or user operation. S2) Map the created X11 application window to the screen, use the created X11 application window to accept the drag-and-drop entry message XdndEnter of the Xdnd protocol of the drag source window to obtain the drag data type, and then unmap the created X11 application window from the screen. In this embodiment, after creating an X11 application in the Wayland compositor, the created X11 application window is mapped out. The drag-and-drop source X11 application window will immediately send an XdndEnter event message to the created X11 application window. XdndEnter is the "entry" message in the X window drag-and-drop protocol, indicating that the drag-and-drop operation has entered the scope of the target window. Through the XdndEnter event message, the compositor can obtain the drag data type of the drag source X11 window. After obtaining the drag data type of the drag source window, the created X11 application window is unmapped. S3) Determine the application type of the target window to be dragged. If the application type of the target window to be dragged is a Wayland application, display the created X11 application window on the screen and use the created X11 application window to receive the Xdnd protocol events and X11 application data of the source window to be dragged. In this embodiment, determining the application type of the dragged target window involves listening to mouse and touchscreen movements. If the mouse or touchscreen enters the window surface of the Wayland application, the application type of the dragged target window is a Wayland application; if the mouse or touchscreen enters the window surface of the X11 application, the application type of the dragged target window is an X11 application. When dragged data into the target Wayland window, the created X11 application window is mapped again to receive events such as drag-and-drop position (XdndPosition) and drop-and-drop (XdndDrop) sent by the drag-source X11 window. S4) Convert the Xdnd protocol event messages to the Wayland drag protocol, and then send the Wayland drag protocol event messages and the X11 application data of the drag source window to the drag target window. During dragging, obtain the Wayland protocol drag response messages, convert the Wayland protocol drag response messages to the Xdnd protocol, and send them to the X11 application of the drag source window through the created X11 application window.

[0042] In this embodiment, the protocol conversion specifically involves using XWayland to convert the Xdnd protocol sent by the drag-and-drop source X11 application window into the data drag-and-drop protocol within Wayland. Simultaneously, the Wayland drag-and-drop protocol sent by the target Wayland application window is converted into the Xdnd protocol. This allows the converted event messages to be sent to each other. The specific process is as follows: 1. The X11 application sends a drag-and-drop entry message XdndEnter to the X11 application window created by the compositor. The compositor uses XWayland to convert XdndEnter into a data device entry message wl_data_device.enter in the Wayland protocol and forwards it to the Wayland application. wl_data_device.enter is a message in the Wayland protocol indicating that the drag-and-drop operation has entered the target window. At the same time, when the mouse moves, a data device mouse movement message wl_data_device.motion is sent to the Wayland application. wl_data_device.motion is a message in the Wayland protocol indicating mouse movement and is used to update the mouse position during the drag-and-drop operation. 2. After receiving the data device entry message wl_data_device.enter and the data device mouse movement message wl_data_device.motion, the Wayland application will determine whether the current data should be received. It will then send the data provider set action message wl_data_offer.set_actions and the data provider accept message wl_data_offer.accept to the compositor to indicate whether it should receive the data and the method of reception (copy or move). wl_data_offer.set_actions is the action used in the Wayland protocol to set drag and drop operations. 3. After receiving the data provider set action message wl_data_offer.set_actions and the data provider accept message wl_data_offer.accept, the synthesizer converts them into drag-and-drop status messages XDnDStatus through XWayland, and then returns them to the X11 application through the created X11 window to inform the target window whether it has accepted the message. 4. After the mouse is released, the X11 application sends a drag-and-drop message XdndDrop to the X11 window created by the compositor. The compositor uses XWayland to convert the received XdndDrop into data source sending messages wl_data_source.send and data device drop messages wl_data_device.drop in the Wayland protocol and sends them to the Wayland application. 5. After receiving data, the Wayland application will send a data provider completion message wl_data_offer.finish and a data provider destruction message wl_data_offer.destroy to the synthesizer to indicate that the data reception has been completed. 6. Finally, the synthesizer uses XWayland to convert the received data provider completion message wl_data_offer.finish and data provider destruction message wl_data_offer.destroy into drag-and-drop completion message XdndFinished, and then sends it to the X11 application through the created X11 window to indicate that the reception is complete.

[0043] In step S3 of this embodiment, after determining the application type of the target window to be dragged, the step further includes: if the application type of the target window to be dragged is an X11 application, the application type of the target window to be dragged is a Wayland application.

[0044] This embodiment also proposes a system for dragging and dropping X application data in a Wayland environment, including a drag-and-drop source window and a target window. The application of the drag-and-drop source window is an X11 application, and the application of the target window is either an X11 application or a Wayland application. The system also includes a Wayland synthesizer. The Wayland synthesizer waits for and obtains the X11 protocol declaration information of the drag-and-drop source window, creates a corresponding X11 application window, and then determines the application type of the drag-and-drop target window. If the application type of the drag-and-drop target window is a Wayland application, the created X11 application is used to receive the event messages of the Xdnd protocol of the drag-and-drop source window and the X11 application data of the drag-and-drop source window. Finally, the event messages of the Xdnd protocol are converted into the Wayland drag-and-drop protocol, and then the event messages of the Wayland drag-and-drop protocol and the X11 application data of the drag-and-drop source window are sent to the drag-and-drop target window.

[0045] This embodiment also proposes an apparatus for dragging and dropping X application data in a Wayland environment, including a processor and a storage medium, wherein the storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the method for dragging and dropping X application data in a Wayland environment.

[0046] This embodiment also proposes a computer-readable storage medium storing a computer program thereon, the computer program being executed by a processor to implement the steps of the method for dragging and dropping X application data in the Wayland environment.

[0047] This embodiment also proposes a computer program product, including a computer program that is executed by a processor to implement the steps of the method for dragging and dropping X application data in the Wayland environment.

[0048] In summary, this invention proposes a method and system for dragging and dropping X application data in a Wayland environment. When dragging begins, the Wayland compositor creates an X11 application window of the same size as the screen to receive XdndEnter event messages. After receiving the XdndEnter event message, the created X11 application window is unmapped. Then, mouse and touchscreen movements are monitored to determine if the application has entered the Wayland surface. Once the application has entered the Wayland surface, the created X11 application window is mapped again to receive Xdnd protocol event messages and data. This allows the Wayland compositor to receive drag and drop events and data from X11 applications and send the data to the Wayland application, thus solving the problem of not being able to drag and drop data from X11 applications into Wayland applications within the Wayland environment.

[0049] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A method for dragging and dropping X application data in a Wayland environment, characterized in that, Applied to a Wayland synthesizer, the method includes the following steps: Wait for and obtain the X11 protocol declaration information of the drag source window, create the corresponding X11 application window, map the created X11 application window to the screen, use the created X11 application window to accept the drag-and-drop entry message XdndEnter of the Xdnd protocol of the drag source window to obtain the drag data type, and then unmap the created X11 application window from the screen. Determine the application type of the target window to be dragged. If the application type of the target window is a Wayland application, use the created X11 application window to receive the Xdnd protocol event message and the X11 application data of the source window to be dragged. The application of the source window to be dragged is an X11 application, and the application of the target window is either an X11 application or a Wayland application. The event messages of the Xdnd protocol are converted to the Wayland drag protocol, and then the event messages of the Wayland drag protocol, along with the X11 application data of the drag source window, are sent to the drag target window.

2. The method for dragging and dropping X application data in the Wayland environment according to claim 1, characterized in that, When creating the corresponding X11 application window, specifically, an X11 application window with only Xdnd protocol send and receive functions is created, but without a specific rendering surface and buffer, in order to avoid the created X11 application window being displayed on the screen.

3. The method for dragging and dropping X application data in the Wayland environment according to claim 1, characterized in that, To determine the application type of the target window being dragged, we specifically listen for mouse and touchscreen movements. If the mouse or touchscreen enters the window surface of a Wayland application, the application type of the target window being dragged is a Wayland application. If the mouse or touchscreen enters the window surface of an X11 application, the application type of the target window being dragged is an X11 application.

4. The method for dragging and dropping X application data in the Wayland environment according to claim 3, characterized in that, After determining the application type of the target window to be dragged, the process also includes the following steps: if the application type of the target window to be dragged is an X11 application, then the application type of the target window to be dragged is a Wayland application.

5. The method for dragging and dropping X application data in the Wayland environment according to claim 1, characterized in that, After sending the Wayland drag-and-drop protocol and the X11 application data of the drag source window to the drag target window, it also includes: obtaining the drag response message of the Wayland protocol during dragging, converting the drag response message of the Wayland protocol into the Xdnd protocol, and sending it to the X11 application of the drag source window through the created X11 application window.

6. A system for dragging and dropping X application data in a Wayland environment, characterized in that, The system includes a drag-and-drop source window and a target window. The drag-and-drop source window is an X11 application, and the target window is either an X11 application or a Wayland application. It also includes a Wayland compositor. The Wayland compositor waits for and obtains the X11 protocol declaration information of the drag-and-drop source window, creates a corresponding X11 application window, maps the created X11 application window to the screen, uses the created X11 application window to receive the drag-and-drop entry message XdndEnter from the drag-and-drop source window to obtain the drag data type, then unmaps the created X11 application window from the screen, then determines the application type of the drag-and-drop target window. If the application type of the drag-and-drop target window is a Wayland application, it uses the created X11 application window to receive the Xdnd protocol events and X11 application data from the drag-and-drop source window, finally converts the Xdnd protocol event messages to the Wayland drag-and-drop protocol, and then sends the Wayland drag-and-drop protocol event messages and the X11 application data from the drag-and-drop target window.

7. A device for dragging and dropping X application data in a Wayland environment, characterized in that, The device includes a processor and a storage medium storing a computer program, which is executed by the processor to implement the steps of the method for dragging and dropping X application data in the Wayland environment as described in any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program is executed by a processor to implement the steps of the method for dragging and dropping X application data in the Wayland environment as described in any one of claims 1 to 5.

9. A computer program product, comprising a computer program, characterized in that, The computer program is executed by a processor to implement the steps of the method for dragging and dropping X application data in the Wayland environment as described in any one of claims 1 to 5.

Images