Input device sharing method and system based on multiple hosts

Abstract

The invention relates to the technical field of computer control, and discloses an input device sharing method and system based on multiple hosts, and the system comprises a host sensing module which is used for determining a sharing end host and a use end host; the input equipment redirection module is used for sensing and determining the position of a mouse in the input equipment in the sharing end host, mapping mouse operation to the use end host, and mapping keyboard operation in the input equipment to the same use end host when the mouse operation is mapped to the use end host; the clipboard sharing module is used for acquiring clipboard contents in the sharing end host and sharing the clipboard contents to the use end host; the file dragging module is used for acquiring files in the sharing end host and transmitting the acquired files to the use end host; the problems that an existing input device sharing system is low in operation efficiency, limited in function and poor in compatibility are solved.

Description

Technical Field

[0001] The present application relates to the field of computer control technology, and in particular to a method and system for sharing input devices based on multiple hosts. Background Art

[0002] In the traditional office environment, a host is usually equipped with a set of keyboard and mouse, and the user's office needs are mostly limited to local operations under this host, such as document editing, spreadsheet processing, etc. These local tasks can be completed only in this host, and do not involve cross-host operations, so the office process is relatively simple. However, with the digitalization and diversification of the office environment, in modern office scenarios, users are often no longer limited to one host device, but need to operate multiple hosts to complete specific tasks. For example, in the process of software development and testing, developers often need to use IDE (Integrated Development Environment) on the development host to edit and write code, and then send the code file to the test host to run the test, and modify the code again on the development host according to the running log information. This frequent cross-host operation usually requires developers to switch back and forth between different hosts. Operations on different hosts also rely on different keyboards and mice, and file and data transmission between hosts need to rely on network communication. In the workflow, developers not only need to frequently manually switch input devices such as keyboards and mice to ensure that the host can correctly respond to input operations, but also easily interrupt the developer's thinking and work efficiency is low. Therefore, it is necessary to use a set of input devices to control multiple different hosts to reduce the redundant processes caused by developers frequently switching input devices in the workflow. At present, the existing methods for multiple hosts to share a set of input devices are mainly divided into hardware solutions and software solutions. The hardware solution mainly includes KVM (Keyboard Video Mouse, multi-computer switching) switch and copy cable. KVM switch can switch the connection between input devices and hosts, so that when developers need to use a certain host, they can connect the input device to the host through the KVM switch. When they need to use the next host, they connect the same set of input devices to the next host through the KVM switch. Although the effect of controlling multiple hosts with one set of input devices is achieved, in essence, the connection between the input device and multiple hosts is still one-to-one during use, and frequent switching is required during use, which is inefficient. Copy cable transmits between two hosts through the USB protocol. While sharing a set of keyboards and mice, it can also support the mutual transmission of clipboards and files. However, copy cable requires the installation of a specific driver and can only be used between two hosts, which is not suitable for multi-host scenarios. Software solutions include Synergy, Barrier, and ShareMouse. Synergy is a software that supports multiple computers to share a set of keyboards and mice, but it is not open source and does not provide free trials. Users need to purchase it on the official website before using it, which increases costs for some users. Barrier is an open source software developed based on the early Synergy code. It can support multiple hosts to share keyboards and mice, but this version has not been updated for many years and has limited support for new systems and new hardware.ShareMouse provides a personal free version that supports basic mouse and keyboard sharing. However, the free version is limited to interconnecting and controlling up to 2 hosts, does not support clipboard sharing and file dragging, and connects hosts in the same LAN by default, which is not suitable for office scenarios in companies. In addition, most existing software solutions are designed based on the X11 desktop environment, and have serious compatibility issues in the increasingly popular Wayland desktop environment. It can be seen that the existing input device sharing system has problems such as low operating efficiency, limited functions, and poor compatibility with the Wayland desktop environment. Summary of the invention

[0003] The present application provides a multi-host based input device sharing method and system to solve the problems of low operating efficiency, limited functions and poor compatibility in the existing input device sharing system.

[0004] In order to achieve the above objectives, the present application is implemented through the following technical solutions: In a first aspect, the present application provides an input device sharing method based on multiple hosts, using an input device sharing system based on multiple hosts, the system comprising: a host perception module, an input device redirection module, a clipboard sharing module and a file drag module, the method comprising: The host sensing module senses the status information of all hosts in the same local area network, determines the shared end hosts and the used end hosts among all the hosts according to the status information, and generates layout information according to the determined shared end hosts and the used end hosts; Wherein, the layout information is the relative position information between the shared end host and the used end host; Determine the position of the mouse in the input device in the shared end host through the input device redirection module, map the mouse in the shared end host to the user end host, and when the mouse is mapped to the user end host, map the keyboard in the input device to the same user end host; The clipboard sharing module is used to obtain the clipboard content in the shared end host, and when the mouse is mapped to the user end host, the clipboard content is shared to the same user end host; The file drag module is used to obtain files in the shared end host, and when the mouse is mapped to the user end host, the obtained files are transferred to the same user end host.

[0005] Optionally, the system further includes: a host layout management module, and the method further includes: The host layout management module is used to read the status information and the layout information in the host perception module, and the online status and relative positions of all hosts are visualized according to the status information and the layout information, and the layout information of all hosts is adjusted according to the visualization results.

[0006] Optionally, the host sensing module senses the status information of all hosts in the same local area network, including: Connect the host perception module to the local area network where all current hosts are located, and use the host perception module to send status detection information to all hosts in the local area network. After receiving the status detection information, the host generates status reply information based on its own status and sends it to the host perception module. The host perception module perceives the status information of all hosts in the same local area network based on the status reply information.

[0007] Optionally, determining a shared end host and a used end host among all hosts according to the state information includes: The host in the local area network detects whether the host is connected to an input device upon receiving the status detection information, and generates first status reply information when the host detects that the input device is connected, and generates second status reply information when the host detects that the input device is not connected; The host perception module is used to judge the status reply information. When it is determined that the status information sent by the host is the first status reply information, the host is determined as a shared end host. When it is determined that the status information sent by the host is the second status reply information, the host is determined as a used end host.

[0008] Optionally, determining the position of the mouse in the input device in the shared end host through the input device redirection module, and mapping the mouse in the shared end host to the user end host, including: Constructing a shared-end desktop coordinate system according to the desktop resolution of the shared-end host, and determining the shared-end mouse coordinate information of the mouse in the input device in the shared-end host desktop in the shared-end desktop coordinate system; Determine the shared end boundary coordinate set of the shared end host desktop boundary in the shared end desktop coordinate system, and when the shared end mouse coordinate information is in the shared end boundary coordinate set, map the mouse operation in the shared end host to the user end host according to the shared end desktop boundary position and the layout of all hosts.

[0009] Optionally, map the mouse on the shared host to the user host, including: Obtaining the operation status of the mouse in the shared host, wherein the operation status includes: mouse acceleration is not enabled and mouse acceleration is enabled; When mouse acceleration is not enabled, the global position of the mouse at the initial moment before the displacement of the shared desktop coordinate system and the relative displacements dx and dy of the mouse on the x-axis and y-axis of the shared desktop coordinate system are obtained, and the updated global position of the mouse after the displacement is calculated using the first global position calculation formula, wherein the first global position calculation formula satisfies the following relationship: ; In the formula, , Indicates the global position of the mouse updated after the displacement occurs. , Indicates the global position of the mouse at the initial moment before the displacement occurs. Indicates the current mouse pointer speed, dx indicates the relative displacement of the mouse on the x-axis, and dy indicates the relative displacement of the mouse on the y-axis; When mouse acceleration is turned on, the global position of the mouse at the initial moment before the displacement of the shared desktop coordinate system and the relative displacement dx and dy of the mouse on the x-axis and y-axis of the shared desktop coordinate system are obtained, and the updated global position of the mouse after the displacement is calculated by the second global position calculation formula, wherein the second global position calculation formula satisfies the following relationship: ; In the formula, dx represents the relative displacement of the mouse on the x-axis, and dy represents the relative displacement of the mouse on the y-axis; Get the resolution of the shared desktop, and calculate the relative position of the mouse to the shared screen based on the resolution of the shared desktop and the global position. The calculation formula satisfies the following relationship: ; In the formula, , Indicates the relative position of the mouse to the shared screen. , Indicates the resolution of the shared screen; The position of the mouse on the user's desktop is calculated based on the relative position and the resolution of the user's desktop. The calculation method satisfies the following relationship: ; In the formula, , Indicates the position of the mouse on the desktop of the user. , Indicates the resolution of the user's screen.

[0010] Optionally, when the mouse is mapped to the end host, the keyboard in the input device is mapped to the same end host, including: Generate virtual input devices in all hosts, and activate corresponding virtual input devices according to the position of the mouse. When the mouse is located in the shared host, activate the corresponding virtual mouse and virtual keyboard in the shared host. When the mouse is located in the user host, activate the corresponding virtual mouse and virtual keyboard in the user host. Control the virtual mouse through the mouse in the input device to perform operations, and control the virtual keyboard through the keyboard in the input device to perform operations.

[0011] Optionally, the clipboard sharing module is used to obtain the clipboard content in the sharing end host, and the clipboard content is shared to the using end host, including: After executing the copy operation in the first process of the shared host, move the mouse to the second host. At this time, a clipboard content management thread will be created in the shared host, and a connection will be established with the synthesizer. At the same time, a paste request will be sent to the synthesizer, and the event sent by the first process will be monitored. After receiving the request, the synthesizer on the shared end host notifies the first process to provide the clipboard content, and the first process sends the clipboard content to the clipboard content management thread of the shared end host; After obtaining the clipboard content, the clipboard content management thread of the shared host will first encrypt the content and then send it to the clipboard sharing module of the second host through the network. There is also a clipboard content management thread in the module. The clipboard content management thread of the end host is used to first decrypt the data, and then simulate the user to perform a copy operation on the end host. The end host synthesizer is used to believe that the copy operation is performed in the clipboard content management thread of the second host. When the second process of the end host performs a paste operation, the synthesizer notifies the clipboard content management thread of the end host, and the clipboard content management thread of the end host sends the clipboard content to the second process of the end host; Use the file drag module to obtain files from the shared host and transfer the obtained files to the user host, including: When you drag and drop files on the shared host and the mouse moves to the edge of the screen, the file drag module will dynamically create a full-screen and transparent auxiliary window on the shared host; When a file is dragged out of the screen of the shared host, the file drag module will simulate the action of releasing the left mouse button on the shared host to end the file dragging process of the shared host. At the same time, the file drag module obtains the path and file name of the file just dragged, and sends the file to the specified directory in the user host; When you drag a file to the end host, the file drag module of the end host will create a full-screen and transparent auxiliary interface on the end host. At this time, the mouse is located in the auxiliary interface. The file drag module simulates the action of pressing the left button of the mouse in the auxiliary interface of the end host, triggers the file drag event initiated by the auxiliary interface of the end host, and at the same time, sets the file transmitted by the shared end host as the source of the dragged file.

[0012] In a second aspect, the present application further provides an input device sharing system based on multiple hosts, the system comprising: The host sensing module is used to sense the status information of all hosts in the same local area network and determine the shared end hosts and the used end hosts among all the hosts; An input device redirection module is used to sense and determine the position of the mouse in the input device in the shared end host, and map the mouse operation to the user end host, and is also used to map the keyboard operation in the input device to the same user end host when mapping the mouse operation to the user end host; The clipboard sharing module is used to obtain the clipboard content in the shared end host and share the clipboard content to the user end host; The file drag module is used to obtain files from the shared host and transfer the obtained files to the user host.

[0013] Optionally, the system further includes: a host layout management module; The host layout management module is used to read the status information in the host perception module, and to visualize the online status of all hosts according to the status information, and to adjust the layout information of all hosts.

[0014] Beneficial effects: The input device sharing system based on multiple hosts provided by the present application realizes the sharing of input devices between multiple hosts and is compatible with two desktop environments, X11 and Wayland. It enables multiple users to operate the same set of input devices through the sharing end, effectively solving the problem of frequent switching of input devices in a multi-host office environment. Through the visual interface, users are supported to drag and drop to adjust the relative position between hosts, and the system can dynamically update the layout, which greatly improves the operational flexibility in a multi-host environment. Through mouse screen boundary detection and input device redirection, the system can automatically switch the focus of the mouse and keyboard, achieve smooth and seamless switching between multiple hosts, avoid the cumbersome operation of manual switching, and improve the user experience. With the switching of the mouse focus, the clipboard content can also be automatically synchronized to ensure the convenience of copying and pasting operations between different hosts. At the same time, the system supports directly dragging files from one host to another to achieve efficient cross-host file sharing. Finally, the present application starts from the bottom layer of the operating system, optimizes the data transmission path, and ensures that the transmission delay of input events is extremely low when switching between hosts, thereby ensuring efficient input response and improving work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A flowchart of a multi-host input device sharing method according to a preferred embodiment of the present application; Figure 2 A module framework diagram of a multi-host input device sharing system according to a preferred embodiment of the present application; Figure 3A diagram showing the relationship between the perception module and the host layout management module in a preferred embodiment of the present application; Figure 4 A schematic diagram of the host layout management of a preferred embodiment of the present application; Figure 5 This is a workflow diagram of an input device redirection module of a preferred embodiment of the present application; Figure 6 This is a schematic diagram of input device redirection according to a preferred embodiment of the present application. DETAILED DESCRIPTION

[0016] The technical solution of the present application is described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0017] Unless otherwise defined, the technical terms or scientific terms used in this application should be understood by people with ordinary skills in the field to which this application belongs. The "first", "second" and similar words used in this application do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, similar words such as "one" or "one" do not indicate quantity restrictions, but indicate the existence of at least one. Similar words such as "connect" or "connected" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship also changes accordingly.

[0018] See also Figure 1 The embodiment of the present application provides an input device sharing method based on multiple hosts, and applies an input device sharing system based on multiple hosts. The system includes: a host perception module, an input device redirection module, a clipboard sharing module, and a file drag module. The method includes: The host sensing module senses the status information of all hosts in the same local area network, determines the shared end hosts and the used end hosts among all the hosts according to the status information, and generates layout information according to the determined shared end hosts and the used end hosts; Wherein, the layout information is the relative position information between the shared end host and the used end host; Determine the position of the mouse in the input device in the shared end host through the input device redirection module, map the mouse in the shared end host to the user end host, and when the mouse is mapped to the user end host, map the keyboard in the input device to the same user end host; The clipboard sharing module is used to obtain the clipboard content in the shared end host, and when the mouse is mapped to the user end host, the clipboard content is shared to the same user end host; The file drag module is used to obtain files in the shared end host, and when the mouse is mapped to the user end host, the obtained files are transferred to the same user end host.

[0019] Optionally, the system further includes: a host layout management module, and the method further includes: The host layout management module is used to read the status information and the layout information in the host perception module, and the online status and relative positions of all hosts are visualized according to the status information and the layout information, and the layout information of all hosts is adjusted according to the visualization results.

[0020] like Figure 2 As shown, in the above embodiment, the input device sharing system mainly includes five functional modules: a host perception module, a host layout management module, an input device redirection module, a clipboard sharing module and a file drag module. The specific functions of these five functional modules are as follows: (1) Host perception module The host sensing module performs host sensing detection through network broadcasting, thus achieving fast and seamless host discovery. Users do not need to manually configure host information, the system can automatically sense other hosts in the same LAN and automatically complete the connection, which greatly simplifies the configuration process and improves the user experience.

[0021] After determining the shared host and the user host, the host sensing module monitors the status information sent by the user, including three status types: "online", "login" and "logout". Among them, online means that the user is currently online and was also online at the last moment; login means that the user is currently online but was offline at the last moment; logout means that the user is currently offline but was online at the last moment.

[0022] Each user sends its status information to the sharer when starting and exiting, informing the sharer of its own status changes. At other times, the sharer broadcasts information to obtain the current status of the user.

[0023] After receiving the status information sent by the user, the sharing end will parse it. The parsed information will include the IP address and current status of the user, and these data will be serialized and stored for use by the host layout management module.

[0024] (2) Host layout management module The main function of the host layout management module is to visualize the status of each user end in the host perception module. The relationship diagram is as follows: Figure 3 As shown, it supports changing the relative layout of hosts in the system by dragging and dropping. This module provides users with an intuitive way to adjust the relative position layout between hosts to adapt to different usage scenarios. The host layout management principle of the preferred embodiment of this application is as follows Figure 4 As shown. After adjusting the position of the user host by dragging, the layout information will be updated and saved in the layout configuration file. The specific fields include the IP address and relative position information of the user (located in one of the upper, lower, left, or right positions of the sharing end). The sharing end will read the layout status of each user end through this file and reflect it in the system in real time.

[0025] The shared end uses the struct servant_device structure to encapsulate and abstract each user end. This structure contains the following fields: The IP address of the user; The socket handle for communicating with the user; The layout information of the shared end is abstracted through struct layout. This structure represents the current host layout and points to the corresponding user host through member functions up, down, left and right. In this way, the relative position of each user can be easily maintained.

[0026] In order to ensure the real-time update of the layout configuration file, the inotify mechanism is used to monitor the file. When the configuration file is modified, the system will trigger the corresponding processing function to update the up, down, left and right pointers in the struct layout to point to the new user end position. Through this mechanism, the system can immediately reflect the changes after the user changes the host layout, keeping the layout of the shared end and the user end synchronized.

[0027] (3) Input device redirection module The input device redirection module is responsible for mapping mouse operations to the end host. When mapping mouse operations to the end host, the keyboard operations in the input device are mapped to the same end host. The entire module process is as follows: Figure 5 shown.

[0028] ⒈Mouse screen boundary detection The trigger event for input device redirection is when the mouse reaches the screen boundary, which is usually achieved by obtaining the global position of the mouse on the screen and comparing it with the screen resolution. In the X11 environment, the absolute position of the mouse can be obtained through the XQueryPointer interface. However, Wayland does not provide an interface for obtaining the global position of the mouse for security and privacy reasons. Therefore, the detection of the mouse screen boundary becomes a difficulty. This application proposes a mouse boundary detection scheme based on relative displacement, which is compatible with X11 and Wayland desktop environments. This scheme sets the mouse position to a reference point on the screen (such as the center of the screen) when the system is initialized. Depending on whether the mouse acceleration option is turned on in the operating system, different calculation methods are used to obtain the global position coordinates of the mouse after the move operation occurs. .

[0029] ① Mouse acceleration is not enabled When the operating system does not enable the mouse acceleration option, the system obtains the mouse position through the libinput_event_pointer_get_dx_unaccelerated and libinput_event_pointer_get_dy_unaccelerated functions in the libinput library. Axis and Relative displacement in the axial direction and . Then the global position of the mouse is calculated by the following formula : ; In the formula, , Represents the global position of the mouse, Indicates the current mouse pointer speed, dx indicates the relative displacement of the mouse on the x-axis, and dy indicates the relative displacement of the mouse on the y-axis, which can be obtained through the gsettings command.

[0030] ② Turn on mouse acceleration When the operating system turns on the mouse acceleration option, the system obtains the mouse position through the libinput_event_pointer_get_dx and libinput_event_pointer_get_dy functions in the libinput library. Axis and Relative displacement in the axial direction and , and calculate the global position of the mouse using the following formula: ; In the formula, , Represents the global position of the mouse, dx represents the relative displacement of the mouse on the x-axis, and dy represents the relative displacement of the mouse on the y-axis.

[0031] When determining the global position of the mouse, since the mouse will move continuously when working, it is necessary to continuously iteratively update the global position of the mouse. This iterative update process can regard the operation of the mouse as countless complete movement processes. In the first movement, the global position of the mouse before moving is used as the global position of the mouse at the initial moment before the displacement occurs, and the global position of the mouse after moving is the global position of the mouse updated after the displacement occurs after the first iterative update. Then, in the second movement, the global position of the mouse updated after the displacement occurs after the iterative update after the previous movement is used as the global position of the mouse at the initial moment before the displacement occurs in this movement. After the second movement is completed, the mouse position at this time is the global position of the mouse updated after the displacement occurs after the second iterative update, and so on. The subsequent movements can iteratively update the global position of the mouse according to the above situation, and the iteratively updated global position calculation can be calculated by the above two calculation formulas.

[0032] For example, before the first movement, assuming that the mouse position is (0,0), and the purpose of the first movement is to click the application at position (3,4), the mouse position can be iteratively updated through the above two calculation formulas according to whether mouse acceleration is turned on. At this time, the global position of the mouse updated after the displacement is (3,4), while the global position of the mouse at the initial moment before the displacement is (0,0). After the first movement is completed and the application is clicked, a new function button appears after the application responds to the mouse click, and the position of the new function button appears at (8,7). At this time, if you want to click the new function button, you need to move the mouse for the second time. In the second movement, the global position of the mouse at the initial moment before the displacement is the global position of the mouse updated after the displacement after the previous movement, that is, (3,4), and the global position of the mouse updated after the displacement in this movement is (8,7). The global position iteratively updated after the previous movement is used as the initial moment position before the next movement to complete the iterative update of the global position of the mouse movement.

[0033] The above examples are for illustration only and are not intended to be limiting.

[0034] ⒉Set the initial mouse position on the target host When the mouse reaches the edge of a host screen, the system calculates the ratio of the mouse's current position coordinates to the screen resolution, and sends the adjacent host's position information to the target host so that the target host can correctly calculate where the mouse should enter and set its position.

[0035] For example, when the shared host and the user host are in a horizontally adjacent layout, in the layout information, the shared host is located on the left side of the user host, and the user host is located on the right side of the shared host. Assuming that the screen resolution of the shared host is , using the host's screen resolution as When the mouse reaches the right edge of the screen on the sharing end, the system determines that the mouse information needs to be sent to the user end, and then performs the following operations: ①The shared end sends mouse information The system calculates the relative position of the mouse relative to the shared screen as follows: ; Then the relative coordinates and layout information (ie, the sharing end is located on the left side of the using end, and the using end is located on the right side of the sharing end) are sent to the using end.

[0036] ② The user receives and processes After receiving the message, the host on the user end first knows that it is located to the right of the sharing end. Then, based on the relative coordinates sent by the sharing end and the resolution of its own screen, it calculates the initial position of the mouse on its own screen (that is, the initial position of the mouse on the user end screen) using the following formula.

[0037] ; In the formula, , Indicates the position of the mouse on the desktop of the user. , Indicates the relative position of the mouse to the shared screen. , Indicates the resolution of the user's screen.

[0038] For the convenience of calculation, when constructing the screen coordinate system, the point at the lower left corner of the screen is usually used as the origin of the coordinate system. Then, when the mouse enters the current screen from the left end or the bottom end of the current screen, one of the x-axis or y-axis of the mouse position must be 0, but the x-axis or y-axis position obtained by calculation is not 0. At this time, the entry position of the mouse in the current screen will be determined according to the layout information. When the mouse enters the current screen from the bottom end of the current screen, Assign a value of 0. When the mouse enters the current screen from the left end of the current screen, Similarly, when the mouse enters the current screen from the top or right end, the calculated x-axis or y-axis position is 0, but the position after entering the screen is not 0. At this time, the mouse entry position in the current screen will be determined according to the layout information. At the same time, the position of the top screen in the y-axis and the position of the right screen in the x-axis will also be determined. When the mouse enters the current screen from the right end, the position of the right screen in the x-axis is assigned to , when the mouse enters the current screen from the top, assign the position of the top screen in the y-axis to , to ensure that mouse operations can be mapped normally.

[0039] According to the calculated Set the initial position of the mouse. The specific setting method is as follows: Create a virtual mouse device that supports absolute position events on the target host. For detailed steps, see the input device virtualization section later in this article.

[0040] The calculated , Position events are written sequentially to the virtual mouse device, and a synchronization event (SYN) is subsequently written to complete the position setting.

[0041] ⒊ Input event redirection When the mouse moves to the user host, the shared end needs to intercept the keyboard and mouse behavior data (collectively referred to as events) and forward it to the user end. The user end injects the events into the pre-virtualized input device to achieve the effect of input device redirection. Different processing strategies are required for different host roles (sharing, use). For example, when the mouse is on the shared host, the system does not need to redirect the keyboard and mouse, and all input operations act directly on the shared host. When the mouse moves to the user host, the shared end needs to forward device events and disable the device. When the mouse moves back, it needs to stop forwarding events and cancel disabling the device. In order to simplify operations and avoid manually handling the different roles of the shared end and the user end, the device redirection module decouples the input device from the shared host. At this time, there is no physical input device for each host. Subsequently, an input device is virtualized in both the user and shared ends, and the concept of input device focus is introduced to indicate which host the input device should act on, such as Figure 6 The schematic diagram of the input device redirection module is shown below.

[0042] ① Input device virtualization Device virtualization enables upper-layer applications to "believe" that there are physical input devices, thereby ensuring their normal operation. Although virtualization-related work usually requires modifications at the kernel level, Linux provides a kernel driver uinput to simplify this process. uinput allows user space programs to create virtual input device nodes and report input events from the keyboard and mouse. The specific virtualization process of the input device is as follows: a: Open the / dev / uinput device file, obtain the file handle vir_fd, and set the basic properties of the virtual device to be created, such as the virtual device name, version number, manufacturer, etc.

[0043] b: Configure detailed properties for the virtual device through ioctl system calls. Configure the event types supported by the virtual device to key events, relative displacement events, and synchronization events. At the same time, by setting UI_SET_KEYBIT, the virtual input device can receive key input events from the keyboard and the left, middle, and right buttons of the mouse. By setting UI_SET_RELBIT, the virtual input device can receive relative displacement events from the mouse in the x-axis and y-axis directions and the relative displacement events generated by the scroll wheel.

[0044] c: Register the virtual device to the operating system. After successful registration, a corresponding device file will be generated. Through the device file handle vir_fd, you can use the write system call to write events to the device, thereby simulating the input operation of the real input device.

[0045] ②Decoupling of shared input devices The decoupling of the input device is achieved by simulating the removal of the device file corresponding to the input device, which makes the operating system mistakenly believe that the input device has been removed, resulting in temporary failure of the device. The specific command is as follows: udevadm trigger --action=remove / dev / input / eventX; Among them, eventX refers to the device file corresponding to the input device, which is usually located in the / dev / input / directory. Therefore, it is also necessary to dynamically determine the device files corresponding to the keyboard and mouse. The detection method adopted in this application is to detect according to the device information recorded in the / proc / bus / input / devices file. Specifically, it searches for device files containing the kbd and mouse keywords to determine the device files corresponding to the keyboard and mouse.

[0046] ③Input device event forwarding In Linux, the input subsystem is responsible for managing various input devices, such as keyboards and mice. This subsystem encapsulates the data generated by the input device into struct input_event. Therefore, the operating system does not need to care about which specific input device the event comes from, but only needs to report these events to the corresponding device files. The upper-level application then interacts with the device files to obtain the actual data and process it. Therefore, the device file of the input device can be opened through the open system call to obtain input data and send this data to the host where the current input device focus is located.

[0047] When using the open system call to obtain events reported by the input device, an infinite loop is usually used to continuously read the input. This method will occupy a CPU core and seriously affect the performance of the entire system. In order to solve this problem, this application uses the epoll mechanism to add the opened input device file handle to epoll. Since epoll is event-driven, it will be in a dormant state when no events arrive, and will not occupy CPU resources; once data arrives, epoll will be awakened to call the event processing function, and the event processing function will forward the event to the input device redirection module in the target host.

[0048] ④Input device event processing After the input device redirection module in the target host receives the input device event, the module first fills the event into the struct input_event structure, and then calls the write system call to write the filled struct input_event to the virtual input device through the opened file handle vir_fd.

[0049] (4) Clipboard sharing module Copying and pasting the clipboard content involves communication between three processes: the process that performs the copy operation, the synthesizer, and the process that performs the paste operation. In this embodiment, the synthesizer mainly refers to the XServer / Wayland synthesizer. The complete copy and paste process in a single host is as follows: The user performs a copy operation in the first process, and the first process notifies the XServer / Wayland synthesizer; then, the user performs a paste operation in the second process, and the second process requests the clipboard content from the XServer / Wayland synthesizer, and the XServer / Wayland synthesizer notifies the first process. The first process sends the clipboard content to the second process, and the second process receives the content and displays it in the corresponding window position.

[0050] The workflow of the clipboard sharing module is described below by taking an example: a copy operation is performed in a first process of a first host, and a paste operation is performed in a second process of a second host.

[0051] ① After the user performs a copy operation in the first process of the first host, the user moves the mouse to the second host. At this time, the system will create a clipboard content management thread in the first host, establish a connection with the XServer / Wayland synthesizer, initiate a paste request to the XServer / Wayland synthesizer, and start listening to events sent by the first process.

[0052] ② After receiving the request, the XServer / Wayland synthesizer on the first host notifies the first process to provide the clipboard content. The first process sends the clipboard content to the clipboard content management thread of the first host.

[0053] ③ After obtaining the clipboard content, the clipboard content management thread of the first host will first encrypt the content and then send it to the clipboard sharing module of the second host through the network. There is also a clipboard content management thread in this module.

[0054] ④ The clipboard content management thread of the second host first decrypts the data, and then simulates the user to perform a copy operation on the second host. The second host XServer / Wayland synthesizer believes that the user has performed a copy operation in the clipboard content management thread of the second host.

[0055] ⑤ When the user performs a paste operation in the second process of the second host, the XServer / Wayland synthesizer will notify the clipboard content management thread of the second host, and the clipboard content management thread of the second host will send the clipboard content to the second process of the second host.

[0056] (5) File drag module The file dragging operation requires the intervention of the XServer / Wayland synthesizer. Other processes cannot obtain the information of the dragged file (such as the file name, file path, etc.). The XServer / Wayland synthesizers between multiple hosts are independent individuals, so it is impossible to directly implement file dragging between multiple hosts. This application proposes a method for creating an auxiliary window as a proxy for relay dragging. For the convenience of explanation, take dragging a file from the first host to the second host as an example to illustrate the working principle and process of the file dragging module.

[0057] ① First, the user drags a file on the first host. When the mouse moves to the edge of the screen, the file drag module dynamically creates a full-screen and transparent auxiliary window on the first host.

[0058] ② When the user drags the file out of the screen of the first host, the file drag module will simulate the action of releasing the left mouse button on the first host, ending the file dragging process of the first host. At the same time, the file drag module obtains the path and file name of the file just dragged, and sends the file to the specified directory in the second host.

[0059] ③ When the user drags the file to the second host, the file drag module of the second host will create a full-screen and transparent auxiliary interface on the second host. At this time, the mouse is located in the auxiliary interface.

[0060] ④ The file drag module simulates the action of pressing the left mouse button in the second host auxiliary interface, triggering the file drag event initiated by the second host auxiliary interface. At the same time, the file transmitted by the first host is set as the source of the dragged file.

[0061] The embodiment of the present application also provides an input device sharing system based on multiple hosts, the system comprising: The host sensing module is used to sense the status information of all hosts in the same local area network and determine the shared end hosts and the used end hosts among all the hosts; An input device redirection module is used to sense and determine the position of the mouse in the input device in the shared end host, and map the mouse operation to the user end host, and is also used to map the keyboard operation in the input device to the same user end host when mapping the mouse operation to the user end host; The clipboard sharing module is used to obtain the clipboard content in the shared end host and share the clipboard content to the user end host; The file drag module is used to obtain files from the shared host and transfer the obtained files to the user host.

[0062] Optionally, the system further includes: a host layout management module; The host layout management module is used to read the status information in the host perception module, and to visualize the online status of all hosts according to the status information, and to adjust the layout information of all hosts.

[0063] The preferred specific embodiments of the present application are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present application without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art based on the concept of the present application through logical analysis, reasoning or limited experiments on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A method for sharing input devices based on multiple hosts, using a system for sharing input devices based on multiple hosts, characterized in that: The system includes: a host perception module, an input device redirection module, a clipboard sharing module and a file drag module, and the method includes: The host sensing module senses the status information of all hosts in the same local area network, determines the shared end hosts and the used end hosts among all the hosts according to the status information, and generates layout information according to the determined shared end hosts and the used end hosts; Wherein, the layout information is the relative position information between the shared end host and the used end host; Determine the position of the mouse in the input device in the shared end host through the input device redirection module, map the mouse in the shared end host to the user end host, and when the mouse is mapped to the user end host, map the keyboard in the input device to the same user end host; The clipboard sharing module is used to obtain the clipboard content in the shared end host, and when the mouse is mapped to the user end host, the clipboard content is shared to the same user end host; The file drag module is used to obtain files in the shared end host, and when the mouse is mapped to the user end host, the obtained files are transferred to the same user end host.

2. The input device sharing method based on multiple hosts according to claim 1, characterized in that: The system further includes: a host layout management module, and the method further includes: The host layout management module is used to read the status information and the layout information in the host perception module, and the online status and relative positions of all hosts are visualized according to the status information and the layout information, and the layout information of all hosts is adjusted according to the visualization results.

3. The input device sharing method based on multiple hosts according to claim 1, characterized in that: The host sensing module senses the status information of all hosts in the same LAN, including: Connect the host perception module to the local area network where all current hosts are located, and use the host perception module to send status detection information to all hosts in the local area network. After receiving the status detection information, the host generates status reply information based on its own status and sends it to the host perception module. The host perception module perceives the status information of all hosts in the same local area network based on the status reply information.

4. The input device sharing method based on multiple hosts according to claim 3, characterized in that: Determining a shared end host and a used end host among all hosts according to the state information includes: The host in the local area network detects whether the host is connected to an input device upon receiving the status detection information, and generates first status reply information when the host detects that the input device is connected, and generates second status reply information when the host detects that the input device is not connected; The host perception module is used to judge the status reply information. When it is determined that the status information sent by the host is the first status reply information, the host is determined as a shared end host. When it is determined that the status information sent by the host is the second status reply information, the host is determined as a used end host.

5. The input device sharing method based on multiple hosts according to claim 1, characterized in that: The input device redirection module is used to determine the position of the mouse in the input device in the shared host, and the mouse in the shared host is mapped to the user host, including: Constructing a shared-end desktop coordinate system according to the desktop resolution of the shared-end host, and determining the shared-end mouse coordinate information of the mouse in the input device in the shared-end host desktop in the shared-end desktop coordinate system; Determine the shared end boundary coordinate set of the shared end host desktop boundary in the shared end desktop coordinate system, and when the shared end mouse coordinate information is in the shared end boundary coordinate set, map the mouse operation in the shared end host to the user end host according to the shared end desktop boundary position and the layout of all hosts.

6. The input device sharing method based on multiple hosts according to claim 5, characterized in that: Map the mouse in the shared host to the user host, including: Obtaining the operation status of the mouse in the shared host, wherein the operation status includes: mouse acceleration is not enabled and mouse acceleration is enabled; When mouse acceleration is not enabled, the global position of the mouse at the initial moment before the displacement of the shared desktop coordinate system and the relative displacements dx and dy of the mouse on the x-axis and y-axis of the shared desktop coordinate system are obtained, and the updated global position of the mouse after the displacement is calculated using the first global position calculation formula, wherein the first global position calculation formula satisfies the following relationship: ； In the formula, , Indicates the global position of the mouse updated after the displacement occurs. , Indicates the global position of the mouse at the initial moment before the displacement occurs. Indicates the current mouse pointer speed, dx indicates the relative displacement of the mouse on the x-axis, and dy indicates the relative displacement of the mouse on the y-axis; When mouse acceleration is turned on, the global position of the mouse at the initial moment before the displacement of the shared desktop coordinate system and the relative displacement dx and dy of the mouse on the x-axis and y-axis of the shared desktop coordinate system are obtained, and the updated global position of the mouse after the displacement is calculated by the second global position calculation formula, wherein the second global position calculation formula satisfies the following relationship: ； In the formula, dx represents the relative displacement of the mouse on the x-axis, and dy represents the relative displacement of the mouse on the y-axis; Get the resolution of the shared desktop, and calculate the relative position of the mouse to the shared screen based on the resolution of the shared desktop and the global position. The calculation formula satisfies the following relationship: ； In the formula, , Indicates the relative position of the mouse to the shared screen. , Indicates the resolution of the shared screen; The position of the mouse on the user's desktop is calculated based on the relative position and the resolution of the user's desktop. The calculation method satisfies the following relationship: ； In the formula, , Indicates the position of the mouse on the desktop of the user. , Indicates the resolution of the user's screen.

7. The input device sharing method based on multiple hosts according to claim 1, characterized in that: When the mouse is mapped to the end host, map the keyboard in the input device to the same end host, including: Generate virtual input devices in all hosts, and activate corresponding virtual input devices according to the position of the mouse. When the mouse is located in the shared host, activate the corresponding virtual mouse and virtual keyboard in the shared host. When the mouse is located in the user host, activate the corresponding virtual mouse and virtual keyboard in the user host. Control the virtual mouse through the mouse in the input device to perform operations, and control the virtual keyboard through the keyboard in the input device to perform operations.

8. The input device sharing method based on multiple hosts according to claim 1, characterized in that: Use the clipboard sharing module to obtain the clipboard content in the shared host and share the clipboard content to the user host, including: After executing the copy operation in the first process of the shared host, move the mouse to the second host. At this time, a clipboard content management thread will be created in the shared host, and a connection will be established with the synthesizer. At the same time, a paste request will be sent to the synthesizer, and the event sent by the first process will be monitored. After receiving the request, the synthesizer on the shared end host notifies the first process to provide the clipboard content, and the first process sends the clipboard content to the clipboard content management thread of the shared end host; After obtaining the clipboard content, the clipboard content management thread of the shared host will first encrypt the content and then send it to the clipboard sharing module of the second host through the network. There is also a clipboard content management thread in the module. The clipboard content management thread of the end host is used to first decrypt the data, and then simulate the user to perform a copy operation on the end host. The end host synthesizer is used to believe that the copy operation is performed in the clipboard content management thread of the second host. When the second process of the end host performs a paste operation, the synthesizer notifies the clipboard content management thread of the end host, and the clipboard content management thread of the end host sends the clipboard content to the second process of the end host; Use the file drag module to obtain files from the shared host and transfer the obtained files to the user host, including: When you drag and drop files on the shared host and the mouse moves to the edge of the screen, the file drag module will dynamically create a full-screen and transparent auxiliary window on the shared host; When a file is dragged out of the screen of the shared host, the file drag module will simulate the action of releasing the left mouse button on the shared host to end the file dragging process of the shared host. At the same time, the file drag module obtains the path and file name of the file just dragged, and sends the file to the specified directory in the user host; When you drag a file to the end host, the file drag module of the end host will create a full-screen and transparent auxiliary interface on the end host. At this time, the mouse is located in the auxiliary interface. The file drag module simulates the action of pressing the left button of the mouse in the auxiliary interface of the end host, triggers the file drag event initiated by the auxiliary interface of the end host, and at the same time, sets the file transmitted by the shared end host as the source of the dragged file.

9. A multi-host based input device sharing system, characterized in that: The system comprises: The host sensing module is used to sense the status information of all hosts in the same local area network and determine the shared end hosts and the used end hosts among all the hosts; An input device redirection module is used to sense and determine the position of the mouse in the input device in the shared end host, and map the mouse operation to the user end host, and is also used to map the keyboard operation in the input device to the same user end host when mapping the mouse operation to the user end host; The clipboard sharing module is used to obtain the clipboard content in the shared end host and share the clipboard content to the user end host; The file drag module is used to obtain files from the shared host and transfer the obtained files to the user host.

10. The input device sharing system based on multiple hosts according to claim 9, characterized in that: The system further comprises: a host layout management module; The host layout management module is used to read the status information in the host perception module, and to visualize the online status of all hosts according to the status information, and to adjust the layout information of all hosts.

Images