A Multi-Host Based Input Device Sharing Method and System

Abstract

The present application relates to the field of computer control technology, and discloses a method and system for sharing input devices based on multiple hosts. The system includes: a host perception module for determining a sharing-end host and a using-end host; an input device redirection module for perceiving and determining the position of the mouse in the input device in the sharing-end host and mapping the mouse operation to the using-end host, and also for mapping the keyboard operation in the input device to the same using-end host when mapping the mouse operation to the using-end host; a clipboard sharing module for obtaining the clipboard content in the sharing-end host and sharing the clipboard content to the using-end host; a file drag-and-drop module for obtaining the files in the sharing-end host and transferring the obtained files to the using-end host. The present application solves the problems of low operation efficiency, limited functions, and poor compatibility existing in the existing input device sharing system.

Description

Technical Field

[0001] This application relates to the field of computer control technology, and particularly to a method and system for sharing input devices based on multiple hosts. Background Art

[0002] In a traditional office environment, usually one host is equipped with a set of keyboard and mouse. Most of the user's office needs are limited to local operations under this host, such as document editing, spreadsheet processing, etc. Completing these local tasks only requires operating within this host and does not involve cross-host operations. Therefore, the office process is relatively simple. However, with the digital and diversified development of the office environment, in modern office scenarios, users are often no longer limited to one host device, but need to operate multiple hosts to complete specific tasks. For example, during software development and testing, developers often need to use an IDE (Integrated Development Environment) on the development host to edit and write code, and then send the code file to the test host to run tests. And based on the running log information, they need to modify the code again on the development host. This frequent cross-host operation usually requires developers to switch back and forth between different hosts. Operating on different hosts also relies on different keyboards and mice, and file and data transfer between hosts depends on network communication. In the work process, developers not only need to frequently manually switch input devices such as keyboards and mice to ensure that the host can correctly respond to input operations, which is very likely to interrupt the developers' thinking, but also the work efficiency is low. Therefore, a set of input devices is needed to control multiple different hosts to reduce the redundant processes generated by developers' frequent switching of input devices in the work process. Currently, the existing methods for sharing a set of input devices among multiple hosts are mainly divided into hardware solutions and software solutions. The hardware solutions are mainly KVM (Keyboard Video Mouse) switches and copy cables. KVM switches can switch the connection between the input device and the host. When a developer needs to use a certain host, the input device can be connected to this host through the KVM switch. When needing to use the next host, the same set of input devices can be connected to the next host through the KVM switch. Although it achieves the effect of controlling multiple hosts with a set of input devices, in essence, during use, the connection between the input device and multiple hosts is still one-to-one, and frequent switching is required during use, resulting in low efficiency. Copy cables are used to transfer data between two hosts through the USB protocol. While sharing a set of keyboard and mouse, they can also support the mutual transfer of the clipboard and files. However, copy cables require the installation of specific driver programs and can only be used between two hosts, not suitable for the scenario of multiple hosts. Software solutions include Synergy, Barrier, and ShareMouse, etc. Synergy is a software that supports sharing a set of keyboard and mouse among multiple computers, but it is not open source and does not provide a free trial. Users need to purchase it on the official website before they can use it, which increases the cost for some users. Barrier is an open-source software developed based on the early Synergy code and can support sharing keyboards and mice among multiple hosts, but this version has not been updated for many years, and its support for new systems and new hardware is limited.ShareMouse provides a free personal version that supports basic mouse and keyboard sharing. However, the free version is limited to connecting and controlling a maximum of 2 hosts, does not support clipboard sharing and file dragging, and by default connects hosts on the same local area network together, making it inapplicable to office scenarios in companies. In addition, most existing software solutions are designed based on the X11 desktop environment and have serious compatibility issues in the gradually popular Wayland desktop environment. It can be seen that existing input device sharing systems have problems such as low operation efficiency, limited functions, and poor compatibility with the Wayland desktop environment. Summary of the Invention

[0003] The present application provides a method and system for sharing input devices based on multiple hosts to solve the problems of low operation efficiency, limited functions, and poor compatibility existing in existing input device sharing systems.

[0004] To achieve the above object, the present application is implemented through the following technical solutions:

[0005] In a first aspect, the present application provides a method for sharing input devices based on multiple hosts, which is applied to a system for sharing input devices based on multiple hosts. The system includes: a host perception module, an input device redirection module, a clipboard sharing module, and a file dragging module. The method includes:

[0006] Perceive the status information of all hosts in the same local area network through the host perception module, determine the sharing-end host and the using-end host among all hosts according to the status information, and generate layout information according to the determined sharing-end host and using-end host;

[0007] Among them, the layout information is the relative position information between the sharing-end host and the using-end host;

[0008] Determine the position of the mouse in the input device in the sharing-end host through the input device redirection module, map the mouse in the sharing-end host to the using-end host, and when the mouse is mapped to the using-end host, map the keyboard in the input device to the same using-end host;

[0009] Obtain the clipboard content in the sharing-end host by using the clipboard sharing module, and when the mouse is mapped to the using-end host, share the clipboard content to the same using-end host;

[0010] Obtain the files in the sharing-end host by using the file dragging module, and when the mouse is mapped to the using-end host, transfer the obtained files to the same using-end host.

[0011] Optionally, the system further includes: a host layout management module, and the method further includes:

[0012] The host layout management module reads the status information and the layout information in the host perception module, visually displays the online status and relative positions of all hosts according to the status information and the layout information, and adjusts the layout information of all hosts according to the visual display result.

[0013] Optionally, the host perception module senses the status information of all hosts in the same local area network, including:

[0014] The host perception module is connected to the local area network where all current hosts are located. The host perception module sends status detection information to all hosts in the local area network. After receiving the status detection information, the host generates a status reply information based on its own status and sends it to the host perception module. The host perception module senses the status information of all hosts in the same local area network according to the status reply information.

[0015] Optionally, a shared-end host and a using-end host are determined among all hosts according to the status information, including:

[0016] When a host in the local area network receives the status detection information, it detects whether it is connected to an input device. When it detects that it is connected to an input device, it generates a first status reply information. When it detects that it is not connected to an input device, it generates a second status reply information;

[0017] The host perception module judges the status reply information. When it judges that the status information sent by the host is the first status reply information, the host is determined as the shared-end host. When it judges that the status information sent by the host is the second status reply information, the host is determined as the using-end host.

[0018] Optionally, the input device redirection module determines the position of the mouse in the input device in the shared-end host and maps the mouse in the shared-end host to the using-end host, including:

[0019] Construct a shared-end desktop coordinate system according to the desktop resolution of the shared-end host, and determine the shared-end mouse coordinate information of the mouse in the input device on the desktop of the shared-end host in the shared-end desktop coordinate system;

[0020] Determine the set of shared-end boundary coordinates of the desktop boundary of the shared-end host in the shared-end desktop coordinate system. When the shared-end mouse coordinate information is within the set of shared-end boundary coordinates, map the mouse operation in the shared-end host to the using-end host according to the position of the shared-end desktop boundary and the layout of all hosts.

[0021] Optionally, mapping the mouse in the shared-end host to the using-end host includes:

[0022] Obtain the operation status of the mouse in the shared-end host, where the operation status includes: mouse acceleration not enabled and mouse acceleration enabled;

[0023] When the mouse acceleration is not enabled, obtain the global position of the mouse at the initial moment before the displacement occurs in the shared-end desktop coordinate system and the relative displacements dx and dy of the mouse on the x-axis and y-axis in the shared-end desktop coordinate system, and calculate the updated global position of the mouse after the displacement through the first global position calculation formula, where the first global position calculation formula satisfies the following relationship:

[0024] ;

[0025] In the formula, 、 represent the updated global position of the mouse after the displacement, 、 represent the global position of the mouse at the initial moment before the displacement occurs, represents the current mouse pointer speed, dx represents the relative displacement of the mouse on the x-axis, and dy represents the relative displacement of the mouse on the y-axis;

[0026] When the mouse acceleration is enabled, obtain the global position of the mouse at the initial moment before the displacement occurs in the shared-end desktop coordinate system and the relative displacements dx and dy of the mouse on the x-axis and y-axis in the shared-end desktop coordinate system, and calculate the updated global position of the mouse after the displacement through the second global position calculation formula, where the second global position calculation formula satisfies the following relationship:

[0027] ;

[0028] In the formula, dx represents the relative displacement of the mouse on the x-axis, and dy represents the relative displacement of the mouse on the y-axis;

[0029] Obtain the resolution of the shared-end desktop, and calculate the relative position of the mouse relative to the shared-end screen according to the resolution of the shared-end desktop and the global position, and its calculation formula satisfies the following relationship:

[0030] ;

[0031] In the formula, 、 represent the relative position of the mouse relative to the shared-end screen, 、 represent the resolution of the shared-end screen;

[0032] Calculate the position of the mouse on the using-end desktop based on the relative position and the resolution of the using-end desktop, and its calculation method satisfies the following relationship:

[0033] ;

[0034] In the formula, , represent the position of the mouse on the desktop of the using end, , represent the resolution of the screen of the using end.

[0035] Optionally, when the mouse is mapped to the host of the using end, the keyboard in the input device is mapped to the same host of the using end, including:

[0036] Generate virtual input devices in all hosts, and activate the corresponding virtual input devices according to the position of the mouse. When the position of the mouse is in the shared-end host, activate the corresponding virtual mouse and virtual keyboard in the shared-end host. When the position of the mouse is in the using-end host, activate the corresponding virtual mouse and virtual keyboard in the using-end host. Operate the virtual mouse through the mouse in the input device, and operate the virtual keyboard through the keyboard in the input device.

[0037] Optionally, use the clipboard sharing module to obtain the clipboard content in the shared-end host and share the clipboard content to the using-end host, including:

[0038] After performing a copy operation in the first process of the shared-end host, move the mouse to the second host. At this time, a clipboard content management thread will be created in the shared-end host, establish a connection with the compositor, and at the same time send a paste request to the compositor and start listening for events sent by the first process;

[0039] After the compositor on the shared-end host receives the request, it notifies the first process to provide the clipboard content, and the first process sends the clipboard content to the clipboard content management thread of the shared-end host;

[0040] After the clipboard content management thread of the shared-end host obtains the clipboard content, it will first encrypt the content and then send it to the clipboard sharing module of the second host through the network. There is also a clipboard content management thread in this module;

[0041] The clipboard content management thread of the using-end host first decrypts the data, and then simulates the user to perform a copy operation on the using-end host. The compositor of the using-end host then believes that the copy operation has been performed in the clipboard content management thread of the second host;

[0042] When the second process of the using-end host performs a paste operation, the compositor will notify the clipboard content management thread of the using-end host, and the clipboard content management thread of the using-end host will send the clipboard content to the second process of the using-end host;

[0043] The file drag-and-drop module is used to obtain files in the shared-end host and transfer the obtained files to the using-end host, including:

[0044] Drag a file on the shared-end host. When the mouse moves to the edge of the screen, the file drag-and-drop module will dynamically create a full-screen and transparent auxiliary window on the shared-end host.

[0045] When the file is dragged out of the screen of the shared-end host, the file drag-and-drop module will simulate the action of releasing the left mouse button on the shared-end host to end the file drag-and-drop process on the shared-end host. At the same time, the file drag-and-drop module obtains the file path and file name of the just-dragged file and sends the file to the specified directory in the using-end host.

[0046] When the file is dragged to the using-end host, the file drag-and-drop module of the using-end host will create a full-screen and transparent auxiliary interface on the using-end host. At this time, the mouse is located in the auxiliary interface.

[0047] The file drag-and-drop module simulates the action of pressing the left mouse button in the auxiliary interface of the using-end host to trigger the file drag-and-drop event initiated by the auxiliary interface of the using-end host. At the same time, the file transmitted from the shared-end host is set as the source of the dragged file.

[0048] In a second aspect, the present application also provides a multi-host-based input device sharing system, and the system includes:

[0049] A host perception module, which is used to perceive the status information of all hosts in the same local area network and determine the shared-end host and the using-end host among all hosts.

[0050] An input device redirection module, which is used to perceive and determine the position of the mouse in the input device in the shared-end host and map the mouse operation to the using-end host. It is also used to map the keyboard operation in the input device to the same using-end host when mapping the mouse operation to the using-end host.

[0051] A clipboard sharing module, which is used to obtain the clipboard content in the shared-end host and share the clipboard content to the using-end host.

[0052] A file drag-and-drop module, which is used to obtain files in the shared-end host and transfer the obtained files to the using-end host.

[0053] Optionally, the system further includes: a host layout management module;

[0054] The host layout management module is used to read the status information in the host perception module and visually display the online status of all hosts according to the status information, and is also used to adjust the layout information of all hosts.

[0055] Beneficial effects:

[0056] The input device sharing system based on multiple hosts provided by this application realizes the sharing of input devices among multiple hosts and is compatible with two desktop environments, X11 and Wayland. It enables multiple clients to operate the same set of input devices through a sharing end, effectively solving the problem of frequently switching input devices in a multi-host office environment. With the support of a visual interface for users to drag and adjust the relative positions among hosts, the system can dynamically update the layout, greatly improving the operation flexibility in a multi-host environment. Through mouse screen boundary detection and input device redirection, the system can automatically switch the focus of the mouse and keyboard, achieving smooth seamless switching among multiple hosts, avoiding the cumbersome operation of manual switching, and enhancing the user experience. Along with the switching of the mouse focus, the clipboard content can also be automatically synchronized to ensure the convenience of copy and paste operations between different hosts. At the same time, the system supports directly dragging files from one host to another to achieve efficient cross-host file sharing. Finally, starting from the underlying operating system, this application optimizes the data transmission path to ensure that when switching between hosts, the transmission delay of input events is extremely low, thus guaranteeing efficient input response and improving work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 It is a flowchart of the method for sharing input devices based on multiple hosts according to a preferred embodiment of this application;

[0058] Figure 2 It is a module framework diagram of the input device sharing system based on multiple hosts according to a preferred embodiment of this application;

[0059] Figure 3 It is a relationship diagram between the perception module and the host layout management module according to a preferred embodiment of this application;

[0060] Figure 4 It is a schematic diagram of the principle of host layout management according to a preferred embodiment of this application;

[0061] Figure 5 It is a flowchart of the working process of the input device redirection module according to a preferred embodiment of this application;

[0062] Figure 6 It is a schematic diagram of input device redirection according to a preferred embodiment of this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0063] The technical solutions of this application will be described clearly and completely below. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all of them. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in this application without creative efforts shall fall within the scope of protection of this application.

[0064] Unless otherwise defined, technical terms or scientific terms used in this application shall have the ordinary meanings understood by those of ordinary skill in the art to which this application belongs. The terms "first", "second" and similar words used in this application do not denote any order, quantity or importance, but are only used to distinguish different components. Similarly, words such as "a" or "one" do not denote a quantity limitation, but mean that there is at least one. The terms "connected" or "coupled" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The terms "upper", "lower", "left", "right", etc. are only used to indicate relative positional relationships, and when the absolute position of the object being described changes, the relative positional relationship also changes accordingly.

[0065] Please refer to Figure 1 , an embodiment of this application provides a method for sharing input devices based on multiple hosts, and applies a system for sharing input devices based on multiple hosts. The system includes: a host perception module, an input device redirection module, a clipboard sharing module, and a file drag-and-drop module. The method includes:

[0066] Perceive the status information of all hosts in the same local area network through the host perception module, determine the shared-end host and the using-end host among all hosts according to the status information, and generate layout information according to the determined shared-end host and using-end host;

[0067] Among them, the layout information is the relative position information between the shared-end host and the using-end host;

[0068] Determine the position of the mouse in the input device in the shared-end host through the input device redirection module, map the mouse in the shared-end host to the using-end host, and when the mouse is mapped to the using-end host, map the keyboard in the input device to the same using-end host;

[0069] Obtain the clipboard content in the shared-end host by using the clipboard sharing module, and share the clipboard content to the same using-end host when the mouse is mapped to the using-end host;

[0070] Obtain the files in the shared-end host by using the file drag-and-drop module, and transfer the obtained files to the same using-end host when the mouse is mapped to the using-end host.

[0071] Optionally, the system further includes: a host layout management module, and the method further includes:

[0072] The host layout management module reads the status information and layout information in the host perception module, visualizes the online status and relative positions of all hosts according to the status information and layout information, and adjusts the layout information of all hosts according to the visualization result.

[0073] As Figure 2 shown, in the above embodiment, the input device sharing system mainly includes five functional modules: a host perception module, a host layout management module, an input device redirection module, a clipboard sharing module, and a file drag-and-drop module. The specific functions of these five functional modules are as follows:

[0074] (1) Host perception module

[0075] The host perception module performs host perception detection through network broadcasts to achieve fast and seamless host discovery. Users do not need to manually configure host information. The system can automatically detect other hosts on the same local area network and automatically complete the connection, greatly simplifying the configuration process and improving the user experience.

[0076] After determining the shared-end host and the using-end host, the host perception module listens for the status information sent by the using-end, including three status types: "online", "login", and "logout". Among them, online means that the using-end is currently in an online state and was also in an online state at the previous moment; login means that the using-end is currently in an online state but was in an offline state at the previous moment; logout means that the using-end is currently in an offline state but was in an online state at the previous moment.

[0077] Each using-end sends its status information to the shared-end when starting and exiting, informing the shared-end of its own status changes. At other times, the shared-end broadcasts information to obtain the current status of the using-end.

[0078] After receiving the status information sent by the using-end, the shared-end will parse it. The parsed information will include the IP address and current status of the using-end, and this data will be serialized and stored for use by the host layout management module.

[0079] (2) Host layout management module

[0080] The main function of the host layout management module is to visually display the status of each using-end in the host perception module. Its relationship diagram is as Figure 3 shown, and it supports changing the relative layout of hosts in the system by dragging. This module provides users with an intuitive way to adjust the relative position layout between hosts to adapt to different usage scenarios. The host layout management principle of the preferred embodiment of this application is as Figure 4As shown in the figure. After adjusting the position of the client host by dragging, the layout information will be updated and saved in the layout configuration file. The specific fields include the IP address of the client and the relative position information (one of the upper, lower, left, or right positions relative to the shared host). The shared host will read the layout status of each client through this file and reflect it in the system in real time.

[0081] The shared host uses the struct servant_device structure to encapsulate and abstract each client. This structure contains the following fields:

[0082] The IP address of the client;

[0083] The socket handle for communicating with this client;

[0084] The layout information of the shared host is abstracted through the struct layout. This structure represents the current host layout, and the member functions up, down, left, and right point to the corresponding client hosts. In this way, the relative positions of each client can be easily maintained.

[0085] To ensure the real-time update of the layout configuration file, the inotify mechanism is used to monitor this file. When the configuration file is modified, the system will trigger the corresponding processing function to update the up, down, left, and right pointers in the struct layout to point to the new positions of the clients. Through this mechanism, the system can immediately reflect the changes after the user changes the host layout and keep the layouts of the shared host and the clients synchronized.

[0086] (3)Input device redirection module

[0087] The input device redirection module is responsible for mapping mouse operations to the client host. At the same time, when mapping mouse operations to the client host, it maps keyboard operations in the input device to the same client host. The entire module process is as Figure 5 shown.

[0088] ⒈ Mouse screen boundary detection

[0089] The triggering event for input device redirection is that the mouse reaches the screen boundary, which is usually achieved by obtaining the global position of the mouse on the screen and comparing it with the screen resolution. In the X11 environment, the absolute position of the mouse can be obtained through the XQueryPointer interface. However, for security and privacy reasons, Wayland does not provide an interface to obtain the global position of the mouse. Therefore, detecting the mouse screen boundary becomes a difficult point. This application proposes a mouse boundary detection scheme based on relative displacement, which is compatible with both X11 and Wayland desktop environments. When the system is initialized, the mouse position is set to a reference point on the screen (such as the center of the screen). According to whether the operating system has enabled the mouse acceleration option, different calculation methods are adopted to obtain the global position coordinates of the mouse after a movement operation. .

[0090] ① Mouse acceleration not enabled

[0091] When the operating system does not enable the mouse acceleration option, the system uses the libinput_event_pointer_get_dx_unaccelerated and libinput_event_pointer_get_dy_unaccelerated functions in the libinput library to obtain the relative displacements of the mouse in the axis and axis directions and . Then, the global position of the mouse is calculated through the following formula :

[0092] ;

[0093] In the formula, , represent the global position of the mouse, represents the current mouse pointer speed, dx represents the relative displacement of the mouse in the x-axis, dy represents the relative displacement of the mouse in the y-axis, and is obtained through the gsettings command.

[0094] ② Mouse acceleration enabled

[0095] When the operating system enables the mouse acceleration option, the system uses the libinput_event_pointer_get_dx and libinput_event_pointer_get_dy functions in the libinput library to obtain the relative displacements of the mouse in the axis and axis directions and , and calculates the global position of the mouse through the following formula:

[0096] ;

[0097] In the formula, , represent the global position of the mouse, dx represents the relative displacement of the mouse on the x-axis, and dy represents the relative displacement of the mouse on the y-axis.

[0098] When determining the global position of the mouse, since the mouse moves continuously during operation, it is necessary to continuously iterate and update the global position of the mouse. This iterative update process can regard the mouse operation as countless complete movement processes. When moving for the first time, the global position of the mouse before the move is used as the global position of the mouse at the initial moment before the displacement occurs, and the global position of the mouse after the move is the global position of the mouse updated after the displacement occurs after the first iterative update. Then, when moving for the second time, the global position of the mouse updated after the displacement occurs after the iterative update of the previous move is used as the global position of the mouse at the initial moment before the displacement occurs during this move. After the second move is completed, the current mouse position is the global position of the mouse updated after the displacement occurs after the second iterative update. And so on, the global position of the mouse can be iteratively updated according to the above situation for subsequent moves, and the calculation of the iteratively updated global position can be calculated through the above two calculation formulas.

[0099] Example: Before the first move, assume the position of the mouse is (0, 0). The purpose of the first move is to click on the application program at the position (3, 4). The position of the mouse can be iteratively updated according to the above two calculation formulas based on whether mouse acceleration is enabled. At this time, the global position of the mouse updated after the displacement occurs is (3, 4), and the global position of the mouse at the initial moment before the displacement occurs is (0, 0). After the first move is completed and the application program responds to the mouse click, a new function button appears, and the position of the new function button appears at (8, 7). At this time, if you want to click on the new function button, you need to make a second move of the mouse. The global position of the mouse at the initial moment before the displacement occurs during the second move is the global position of the mouse updated after the displacement occurs after the previous move, that is, (3, 4), and the global position of the mouse updated after the displacement occurs during this move is (8, 7). By using the global position updated after the iterative update of the previous move as the initial moment position before the next move, the iterative update of the global position of the mouse movement can be completed.

[0100] The above example is only for illustrative purposes and is not subject to actual limitations.

[0101] ⒉ Setting the Initial Position of the Mouse on the Target Host

[0102] When the mouse reaches the edge of a host screen, the system calculates the ratio of the current position coordinates of the mouse to the screen resolution and sends the orientation information of the adjacent host to the target host together, so that the target host can correctly calculate where the mouse should enter and set its position.

[0103] For example, when the sharing host and the using host are in a horizontal adjacent layout, in the layout information, at this time the sharing host is on the left side of the using host, and the using host is on the right side of the sharing host. Assume that the screen resolution of the sharing host is , and the screen resolution of the using host is . When the mouse reaches the right screen edge on the sharing host, the system will determine that the mouse information needs to be sent to the using host. At this time, the following operations will be performed:

[0104] ① The sharing host sends the mouse information

[0105] The system calculates the relative position of the mouse relative to the screen of the sharing host. Specifically:

[0106] ;

[0107] Then send this relative coordinate and the layout information (that is, the sharing host is on the left side of the using host, and the using host is on the right side of the sharing host) to the using host.

[0108] ② The using host receives and processes

[0109] After receiving the message, the using host first knows that it is on the right side of the sharing host. Then, according to the relative coordinate sent by the sharing host and the resolution of its own screen, it calculates the initial position of the mouse on its own screen (that is, the initial position of the mouse on the screen of the using host) through the following formula.

[0110] ;

[0111] In the formula, , represent the position of the mouse on the desktop of the using host, , represent the relative position of the mouse relative to the screen of the sharing host, , represent the resolution of the screen of the using host.

[0112] For the convenience of calculation, when constructing the screen coordinate system, usually the point at the lower left corner of the screen is used as the origin of the coordinate system. Then, when the mouse enters the current screen from the left end or the lower end of the current screen, one of the x-axis or y-axis of the mouse position will be 0, but the position of the x-axis or y-axis obtained according to the calculation is not 0. At this time, the entry position of the mouse on the current screen will be determined according to the layout information. When the mouse enters the current screen from the lower end of the current screen, then An assignment is made to set it to 0. When the mouse enters the current screen from the left end of the current screen, then An assignment is made to set it to 0. Similarly, when the mouse enters the current screen from the upper end or the right end, according to the calculated position on the x-axis or y-axis, it is 0, but the position after entering the screen is not 0. At this time, the entry position of the mouse on the current screen will also be determined according to the layout information, and at the same time, the position of the upper screen on the y-axis and the position of the right screen on the x-axis will be determined. When the mouse enters the current screen from the right end, the position of the right screen on the x-axis is assigned to , and when the mouse enters the current screen from the upper end, the position of the upper screen on the y-axis is assigned to , to ensure that mouse operations can be normally mapped.

[0113] According to the calculated Set the initial position of the mouse. The specific setting method is as follows:

[0114] Create a virtual mouse device that supports absolute position events in the target host. For the specific steps, please refer to the input device virtualization part in the following text.

[0115] Write the calculated , position events into this virtual mouse device in sequence, and then write a synchronization event (SYN) to complete the position setting.

[0116] ⒊ Input event redirection

[0117] When the mouse moves to the using-end host, the sharing end needs to intercept and forward the behavior data (collectively referred to as events) of the keyboard and mouse to the using end, and the using end injects the events into the pre-virtualized input device to achieve the effect of input device redirection. Different processing strategies need to be adopted for different host roles (sharing, using). For example, when the mouse is located on the sharing-end host, the system does not need to perform keyboard and mouse redirection operations, and all input operations directly act on the sharing-end host. When the mouse moves to the using-end host, the sharing end needs to forward device events and disable the device. When the mouse moves back, it needs to stop forwarding events and cancel the device disabling. To simplify the operation and avoid manually handling different roles of the sharing end and the using end, the device redirection module decouples the input device from the sharing-end host. At this time, for each host, there is no physical input device. Subsequently, a virtual input device is created in both the using end and the sharing end, and the concept of input device focus is introduced to indicate which host the input device should act on, as shown in Figure 6 the schematic diagram of the input device redirection module.

[0118] ① Input device virtualization

[0119] Device virtualization enables upper-layer application programs to "think" that physical input devices exist, thus ensuring their normal operation. Although work related to virtualization usually requires modification at the kernel layer, Linux provides a kernel driver uinput to simplify this process. Uinput allows programs in user space to create virtual input device nodes and report input events from keyboards and mice. The specific virtualization process of input devices is as follows:

[0120] a: Open the / dev / uinput device file to obtain the file handle vir_fd, and set the basic attributes of the virtual device to be created, such as the virtual device name, version number, manufacturer, etc.

[0121] b: Configure detailed attributes for the virtual device through the ioctl system call. Configure the event types supported by the virtual device as key events, relative displacement events, and synchronization events. At the same time, by setting UI_SET_KEYBIT, the virtual input device can receive key input events from the left, middle, and right buttons of the keyboard and mouse. By setting UI_SET_RELBIT, the virtual input device can receive relative displacement events in the x-axis and y-axis directions of the mouse, as well as relative displacement events generated by the scroll wheel.

[0122] c: Register the virtual device with the operating system. After successful registration, a corresponding device file will be generated. Through the device file handle vir_fd, the write system call can be used to write events to the device, thereby simulating the input operations of real input devices.

[0123] ② Decoupling of input devices at the sharing end

[0124] Decoupling of input devices is achieved by simulating the removal of the device file corresponding to the input device, which makes the operating system mistakenly think that the input device has been removed, resulting in the temporary failure of the device. The specific command is as follows:

[0125] udevadm trigger --action=remove / dev / input / eventX;

[0126] Among them, eventX refers to the device file corresponding to the input device, which is usually located in the / dev / input / directory. Therefore, it is also necessary to dynamically determine the device files corresponding to the keyboard and mouse. The detection method adopted in this application is to detect according to the device information recorded in the / proc / bus / input / devices file. Specifically, it is to find the device files containing the keywords kbd and mouse to determine the device files corresponding to the keyboard and mouse.

[0127] ③ Forwarding of input device events

[0128] In Linux, the input subsystem is responsible for managing various input devices, such as keyboards and mice. This subsystem encapsulates the data generated by input devices into struct input_event. Therefore, the operating system doesn't need to care about which specific input device an event comes from, but only needs to report these events to the corresponding device files, and upper-layer applications can interact with the device files to obtain the actual data and process it. Therefore, the device file of the input device can be opened through the open system call to obtain input data and send this data to the host where the current input device focus is located.

[0129] When using the open system call to obtain events reported by input devices, a dead loop is usually used to continuously read the input. This method will occupy one CPU core and seriously affect the performance of the entire system. To solve this problem, this application uses the epoll mechanism and adds the opened input device file handle to epoll. Since epoll is event-driven, it will be in a dormant state when no events arrive and does not occupy CPU resources; once data arrives, epoll will be awakened to call the event handling function, and the event handling function will forward the event to the input device redirection module in the target host.

[0130] ④ Input device event handling

[0131] After the input device redirection module in the target host receives an input device event, this module first fills the event into the struct input_event structure, and then calls the write system call to write the filled struct input_event to the virtual input device through the opened file handle vir_fd.

[0132] (4)Clipboard sharing module

[0133] The copy and paste of clipboard content involves communication between three processes: the process that executes the copy operation, the compositor, and the process that executes the paste operation. In this embodiment, the compositor mainly refers to the XServer / Wayland compositor. The complete copy and paste process in a single host is as follows:

[0134] The user executes a copy operation in the first process, and the first process will notify the XServer / Wayland compositor; subsequently, the user executes a paste operation in the second process, and the second process requests the clipboard content from the XServer / Wayland compositor. The XServer / Wayland compositor then notifies the first process, and the first process sends the clipboard content to the second process, and the second process receives the content and displays it at the corresponding window position.

[0135] The working process of the clipboard sharing module is illustrated by an example as follows: A copy operation is performed in the first process of the first host, and a paste operation is performed in the second process of the second host.

[0136] ① After the user performs a copy operation in the first process of the first host, the user moves the mouse to the second host. At this time, the system creates a clipboard content management thread in the first host, establishes a connection with the XServer / Wayland compositor, initiates a paste request to the XServer / Wayland compositor, and starts listening for events sent by the first process.

[0137] ② After the XServer / Wayland compositor on the first host receives the request, it notifies the first process to provide the clipboard content. The first process sends the clipboard content to the clipboard content management thread of the first host.

[0138] ③ After the clipboard content management thread of the first host obtains the clipboard content, it first encrypts the content and then sends it to the clipboard sharing module of the second host through the network. There is also a clipboard content management thread in this module.

[0139] ④ The clipboard content management thread of the second host first decrypts the data and then simulates the user performing a copy operation on the second host. The XServer / Wayland compositor on the second host then believes that the user has performed a copy operation in the clipboard content management thread of the second host.

[0140] ⑤ When the user performs a paste operation in the second process of the second host, the XServer / Wayland compositor notifies the clipboard content management thread of the second host, and the clipboard content management thread of the second host sends the clipboard content to the second process of the second host.

[0141] (5) File drag-and-drop module

[0142] File drag-and-drop operations require the interference of the XServer / Wayland compositor. Other processes cannot obtain information about the dragged files (such as file names, file paths, etc.). The XServer / Wayland compositors between multiple hosts are independent entities, so it is impossible to directly implement file drag-and-drop between multiple hosts. This application proposes a method of creating an auxiliary window as a proxy for relay drag-and-drop. For the convenience of explanation, taking the example of dragging a file from the first host to the second host, the working principle and process of the file drag-and-drop module are clarified.

[0143] ① First, the user drags a file on the first host. When the mouse moves to the edge of the screen, the file drag-and-drop module dynamically creates a full-screen and transparent auxiliary window on the first host.

[0144] ② When the user drags a file out of the screen of the first host, the file drag-and-drop module simulates the action of releasing the left mouse button on the first host to end the file drag-and-drop process on the first host. Meanwhile, the file drag-and-drop module obtains the file path and file name of the just-dragged file and sends the file to the specified directory in the second host.

[0145] ③ When the user drags a file to the second host, the file drag-and-drop module on the second host creates a full-screen and transparent auxiliary interface on the second host. At this time, the mouse is located in the auxiliary interface.

[0146] ④ The file drag-and-drop module simulates the action of pressing the left mouse button in the auxiliary interface of the second host to trigger the file drag-and-drop event initiated by the auxiliary interface of the second host. Meanwhile, the file transmitted from the first host is set as the source of the dragged file.

[0147] The embodiment of the present application further provides a multi-host-based input device sharing system, and the system includes:

[0148] A host perception module, configured to perceive the status information of all hosts in the same local area network and determine a shared-end host and a using-end host among all hosts;

[0149] An input device redirection module, configured to perceive and determine the position of the mouse in the input device in the shared-end host and map the mouse operation to the using-end host, and is further configured to map the keyboard operation in the input device to the same using-end host when mapping the mouse operation to the using-end host;

[0150] A clipboard sharing module, configured to obtain the clipboard content in the shared-end host and share the clipboard content to the using-end host;

[0151] A file drag-and-drop module, configured to obtain files in the shared-end host and transmit the obtained files to the using-end host.

[0152] Optionally, the system further includes: a host layout management module;

[0153] The host layout management module is configured to read the status information in the host perception module, visually display the online status of all hosts according to the status information, and is further configured to adjust the layout information of all hosts.

[0154] The preferred specific embodiments of the present application have been described in detail above. It should be understood that those of ordinary skill in the art can make many modifications and variations according to the concept of the present application without creative labor. Therefore, all technical solutions that can be obtained by those skilled in the art in the technical field of the present application based on the concept of the present application through logical analysis, reasoning or limited experiments on the basis of the prior art should be within the protection scope determined by the claims.

Claims

1. A method for sharing input devices based on multiple hosts, using a system for sharing input devices based on multiple hosts, characterized in that: The system includes: a host perception module, an input device redirection module, a clipboard sharing module and a file drag module. The method is applicable to the Wayland environment. The method includes: The host sensing module senses the status information of all hosts in the same local area network, determines the shared end hosts and the used end hosts among all the hosts according to the status information, and generates layout information according to the determined shared end hosts and the used end hosts; Wherein, the layout information is the relative position information between the shared end host and the used end host; Determine the position of the mouse in the input device in the shared end host through the input device redirection module, map the mouse in the shared end host to the user end host, and when the mouse is mapped to the user end host, map the keyboard in the input device to the same user end host; The clipboard sharing module is used to obtain the clipboard content in the shared end host, and when the mouse is mapped to the user end host, the clipboard content is shared to the same user end host; Use the file drag and drop module to obtain files in the shared end host, and when the mouse is mapped to the user end host, transfer the obtained files to the same user end host; The input device redirection module is used to determine the position of the mouse in the input device in the shared host, and the mouse in the shared host is mapped to the user host, including: Constructing a shared-end desktop coordinate system according to the desktop resolution of the shared-end host, and determining the shared-end mouse coordinate information of the mouse in the input device in the shared-end host desktop in the shared-end desktop coordinate system; Determine a shared-end boundary coordinate set of the shared-end host desktop boundary in the shared-end desktop coordinate system in the shared-end desktop coordinate system, and when the shared-end mouse coordinate information is in the shared-end boundary coordinate set, map the mouse operation in the shared-end host to the user host according to the shared-end desktop boundary position and the layout of all hosts; Map the mouse in the shared host to the user host, including: Obtaining the operation status of the mouse in the shared host, wherein the operation status includes: mouse acceleration is not enabled and mouse acceleration is enabled; When mouse acceleration is not enabled, the global position of the mouse at the initial moment before the displacement of the shared desktop coordinate system and the relative displacements dx and dy of the mouse on the x-axis and y-axis of the shared desktop coordinate system are obtained, and the updated global position of the mouse after the displacement is calculated using the first global position calculation formula, wherein the first global position calculation formula satisfies the following relationship: ； In the formula, , Indicates the global position of the mouse updated after the displacement occurs. , Indicates the global position of the mouse at the initial moment before the displacement occurs. Indicates the current mouse pointer speed, dx indicates the relative displacement of the mouse on the x-axis, and dy indicates the relative displacement of the mouse on the y-axis; When mouse acceleration is turned on, the global position of the mouse at the initial moment before the displacement of the shared desktop coordinate system and the relative displacement dx and dy of the mouse on the x-axis and y-axis of the shared desktop coordinate system are obtained, and the updated global position of the mouse after the displacement is calculated by the second global position calculation formula, wherein the second global position calculation formula satisfies the following relationship: ； In the formula, dx represents the relative displacement of the mouse on the x-axis, and dy represents the relative displacement of the mouse on the y-axis; Get the resolution of the shared desktop, and calculate the relative position of the mouse to the shared screen based on the resolution of the shared desktop and the global position. The calculation formula satisfies the following relationship: ； In the formula, , Indicates the relative position of the mouse to the shared screen. , Indicates the resolution of the shared screen; The position of the mouse on the user's desktop is calculated based on the relative position and the resolution of the user's desktop. The calculation method satisfies the following relationship: ； In the formula, , Indicates the position of the mouse on the desktop of the user. , Indicates the resolution of the user's screen.

2. The input device sharing method based on multiple hosts according to claim 1, characterized in that: The system further includes: a host layout management module, and the method further includes: The host layout management module is used to read the status information and the layout information in the host perception module, and the online status and relative positions of all hosts are visualized according to the status information and the layout information, and the layout information of all hosts is adjusted according to the visualization results.

3. The input device sharing method based on multiple hosts according to claim 1, characterized in that: The host sensing module senses the status information of all hosts in the same LAN, including: Connect the host perception module to the local area network where all current hosts are located, and use the host perception module to send status detection information to all hosts in the local area network. After receiving the status detection information, the host generates status reply information based on its own status and sends it to the host perception module. The host perception module perceives the status information of all hosts in the same local area network based on the status reply information.

4. The input device sharing method based on multiple hosts according to claim 3, characterized in that: Determining a shared end host and a used end host among all hosts according to the state information includes: The host in the local area network detects whether the host is connected to an input device upon receiving the status detection information, and generates first status reply information when the host detects that the input device is connected, and generates second status reply information when the host detects that the input device is not connected; The host perception module is used to judge the status reply information. When it is determined that the status information sent by the host is the first status reply information, the host is determined as a shared end host. When it is determined that the status information sent by the host is the second status reply information, the host is determined as a used end host.

5. The input device sharing method based on multiple hosts according to claim 1, characterized in that: When the mouse is mapped to the end host, map the keyboard in the input device to the same end host, including: Generate virtual input devices in all hosts, and activate corresponding virtual input devices according to the position of the mouse. When the mouse is located in the shared host, activate the corresponding virtual mouse and virtual keyboard in the shared host. When the mouse is located in the user host, activate the corresponding virtual mouse and virtual keyboard in the user host. Control the virtual mouse through the mouse in the input device to perform operations, and control the virtual keyboard through the keyboard in the input device to perform operations.

6. The input device sharing method based on multiple hosts according to claim 1, characterized in that: Use the clipboard sharing module to obtain the clipboard content in the shared host and share the clipboard content to the user host, including: After executing the copy operation in the first process of the shared host, move the mouse to the second host. At this time, a clipboard content management thread will be created in the shared host, and a connection will be established with the synthesizer. At the same time, a paste request will be sent to the synthesizer, and the event sent by the first process will be monitored. After receiving the request, the synthesizer on the shared end host notifies the first process to provide the clipboard content, and the first process sends the clipboard content to the clipboard content management thread of the shared end host; After obtaining the clipboard content, the clipboard content management thread of the shared host will first encrypt the content and then send it to the clipboard sharing module of the second host through the network. There is also a clipboard content management thread in the module. The clipboard content management thread of the end host is used to first decrypt the data, and then simulate the user to perform a copy operation on the end host. The end host synthesizer is used to believe that the copy operation is performed in the clipboard content management thread of the second host. When the second process of the end host performs a paste operation, the synthesizer notifies the clipboard content management thread of the end host, and the clipboard content management thread of the end host sends the clipboard content to the second process of the end host; Use the file drag module to obtain files from the shared host and transfer the obtained files to the user host, including: When you drag and drop files on the shared host and the mouse moves to the edge of the screen, the file drag module will dynamically create a full-screen and transparent auxiliary window on the shared host; When a file is dragged out of the screen of the shared host, the file drag module will simulate the action of releasing the left mouse button on the shared host to end the file dragging process of the shared host. At the same time, the file drag module obtains the path and file name of the file just dragged, and sends the file to the specified directory in the user host; When you drag a file to the end host, the file drag module of the end host will create a full-screen and transparent auxiliary interface on the end host. At this time, the mouse is located in the auxiliary interface. The file drag module simulates the action of pressing the left button of the mouse in the auxiliary interface of the end host, triggers the file drag event initiated by the auxiliary interface of the end host, and at the same time, sets the file transmitted by the shared end host as the source of the dragged file.

7. A multi-host based input device sharing system, used to execute the method steps as described in any one of claims 1 to 6, characterized in that: The system comprises: The host sensing module is used to sense the status information of all hosts in the same local area network and determine the shared end hosts and the used end hosts among all the hosts; An input device redirection module is used to sense and determine the position of the mouse in the input device in the shared end host, and map the mouse operation to the user end host, and is also used to map the keyboard operation in the input device to the same user end host when mapping the mouse operation to the user end host; The clipboard sharing module is used to obtain the clipboard content in the shared end host and share the clipboard content to the user end host; The file drag module is used to obtain files from the shared host and transfer the obtained files to the user host.

8. The input device sharing system based on multiple hosts according to claim 7, characterized in that: The system further comprises: a host layout management module; The host layout management module is used to read the status information in the host perception module, and to visualize the online status of all hosts according to the status information, and to adjust the layout information of all hosts.

Images