A cross-device object-dragging method and device

Abstract

The application provides a cross-device object dragging method and device, and relates to the field of electronic devices. The use efficiency of multi-terminal collaborative use is improved, and the interaction between terminals during multi-terminal collaborative use is increased. The specific scheme is as follows: a second terminal displays a first cursor on an object displayed by the second terminal; the second terminal receives a dragging operation input by a user using an input device of a first terminal, and the dragging operation is used to initiate a dragging operation on the object; in response to the dragging operation, the second terminal displays an animation of the object moving with the first cursor on a display screen of the second terminal; when it is determined that the object is dragged out of the edge of the display screen of the second terminal, the second terminal sends dragging data to the first terminal, so that the first terminal continues the dragging.

Description

[0001] This application is a divisional application. The original application has the application number 202080100104.3 and the original application date is December 31, 2020. The entire contents of the original application are incorporated herein by reference. Technical Field

[0002] This application relates to the field of electronic devices, and more particularly to a method and device for dragging and dropping objects across devices. Background Technology

[0003] With the development of electronic technology and mobile internet, a user can simultaneously own multiple terminals such as mobile phones, tablets, personal computers (PCs), and smart home devices (such as televisions). Generally, the use of each terminal is relatively independent. However, in scenarios requiring collaborative use of multiple terminals, such as collaborative office work, users will connect multiple terminals together for joint use. For example, a user with a PC and a mobile phone can connect the PC and phone wirelessly or via wired connection to work together on the same device. Summary of the Invention

[0004] This application provides a cross-device object drag-and-drop method and device. In scenarios where multiple terminals (such as PCs and mobile phones) are used collaboratively, keyboard and mouse sharing technology is used to realize the drag-and-drop of objects such as application icons, application interfaces, files, text or folders from the mobile phone to the PC, which improves the efficiency of PC and mobile phone collaborative use and increases the interaction between terminals when multiple terminals are used collaboratively.

[0005] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0006] A first aspect of this application provides a cross-device object dragging method, which can be applied to a second terminal connected to a first terminal. The method may include: the second terminal displaying a first cursor on an object displayed on the second terminal; the second terminal receiving a drag operation input by a user using an input device of the first terminal, the drag operation being used to initiate dragging of an object; in response to the drag operation, the second terminal displaying an animation of the object moving with the first cursor on the display screen of the second terminal; and the second terminal sending drag data to the first terminal when it determines that the object has been dragged off the edge of the display screen of the second terminal.

[0007] As an example, the input device of the first terminal mentioned above can be a mouse, touchpad, etc. The first cursor can be a cursor displayed on the screen of the second terminal.

[0008] By adopting the above technical solution, in scenarios where multiple terminals are used collaboratively, users can use input devices such as a mouse to drag and drop objects from one terminal to another, following the cursor. For example, a user can use the input device of one terminal to drag and drop objects from one terminal to another, improving the efficiency of multi-terminal collaborative use, increasing the interaction between terminals, and enhancing the user experience.

[0009] In one possible implementation, the drag data can be used by a first terminal to display an animation of an object moving with a second cursor on its screen. The second cursor can be a cursor displayed on the first terminal's screen. By sending the drag data to other terminals, those other terminals can continue displaying the animation of the object moving with the cursor based on user input, thus enabling drag continuation.

[0010] In another possible implementation, the object can be an application icon; or, the object can be a window that includes the application's interface. When the object is an application icon, the drag data can include the application icon; when the object is a window, the drag data can include the application's interface.

[0011] In another possible implementation, after sending drag data to the first terminal, the method may further include: the second terminal receiving an indication message from the first terminal, which is sent by the first terminal after the user inputs a drag-and-release operation using the first terminal's input device, indicating that the dragging of the object has ended; the second terminal sending data to the first terminal, which is used by the first terminal to display the application's interface on the first terminal's screen. After the dragging of the object ends, corresponding data is sent to the first terminal so that the first terminal can display the interface of the application corresponding to the dragged object on its screen.

[0012] In another possible implementation, when the object is an application icon, after the second terminal receives a drag operation input by the user using the input device of the first terminal, the method may further include: the second terminal creating a draggable object based on the application icon, the draggable object including an icon identical to the application icon; in response to the drag operation, the second terminal displaying an animation of the object moving with a first cursor on its display screen, including: in response to the drag operation, the second terminal displaying the animation of the draggable object moving with the first cursor. The created icon, identical to the icon of the dragged application, can be used for dragging.

[0013] In another possible implementation, when the object is a window, before the second terminal displays the first cursor on the object displayed on the second terminal, the method may further include: the second terminal displaying the application's interface; when the first cursor is displayed on the status bar of the second terminal, the second terminal receiving a long press operation input by the user using the input device of the first terminal to trigger the application's interface windowing; in response to the long press operation, the second terminal displaying a window; the window specifically includes a screenshot of the application's interface, and the aforementioned dragged data specifically includes the screenshot of the application's interface.

[0014] In another possible implementation, the second terminal displays an animation of an object moving with the first cursor on the display screen of the second terminal, which may include: the second terminal displays an animation of the shadow of the object moving with the first cursor on the display screen of the second terminal.

[0015] In another possible implementation, the object can be text, a file, or a folder; the drag data includes a bitmap of the drag event content and the shadow; where, when the object is text, the drag event content includes the text, and when the object is a file or folder, the drag event content is the file path.

[0016] In another possible implementation, after the second terminal displays an animation of the object moving with the first cursor on its display screen in response to the drag operation, the method may further include: the second terminal acquiring and caching the drag data.

[0017] In another possible implementation, before sending the drag-and-drop data to the first terminal, the method further includes: the second terminal receiving a request message from the first terminal, the request message being used to request drag-and-drop data; sending the drag-and-drop data to the first terminal may include: the second terminal sending cached drag-and-drop data to the first terminal.

[0018] In another possible implementation, before the second terminal displays the first cursor on the object displayed on the second terminal, the method may further include: the second terminal receiving first shuttle status information from the first terminal, the first shuttle status information indicating the start of shuttle operation; and in response to the first shuttle status information, the second terminal displaying the first cursor on the second terminal. After receiving the shuttle status information indicating the start of shuttle operation, the user can use the input device of the first terminal to control the second terminal.

[0019] In another possible implementation, after the first cursor is displayed on the second terminal, the method may further include: the second terminal receiving an operation input by the user using the input device of the first terminal to move the first cursor; in response to the operation to move the first cursor, the second terminal displaying an animation of the first cursor movement on the display screen of the second terminal until the first cursor moves onto an object. After cursor movement begins, the user can control the second terminal using the input device of the first terminal.

[0020] In another possible implementation, the drag operation described above may include a press operation and a move operation; the second terminal receives the drag operation input by the user using the input device of the first terminal, including: the second terminal receiving a first operation parameter from the first terminal, the first operation parameter being the operation parameter contained in a press event received by the first terminal after the user performs a press operation using the input device of the first terminal; the second terminal receiving a second operation parameter from the first terminal, the second operation parameter being the operation parameter contained in a move event received by the first terminal after the user performs a move operation using the input device of the first terminal; the second terminal simulating a press event according to the first operation parameter and simulating a move event according to the second operation parameter, the press event and the move event being used to indicate the initiation of dragging an object; in response to the drag operation, the second terminal displays an animation of the object moving with the first cursor on the display screen of the second terminal, including: in response to the press event and the move event, the second terminal displays an animation of the object moving with the first cursor.

[0021] In another possible implementation, the second terminal receiving the user's operation of moving the first cursor using the input device of the first terminal may include: the second terminal receiving a third operation parameter from the first terminal, the third operation parameter being the operation parameter contained in the movement event received by the first terminal after the user performs the movement operation using the input device of the first terminal; the second terminal simulating the movement event according to the third operation parameter, the movement event being used to indicate the movement of the first cursor.

[0022] In another possible implementation, after the second terminal determines that the object has been dragged off the edge of its display screen, the method may further include: the second terminal sending second shuttle status information to the first terminal, the second shuttle status information indicating the end of the shuttle. By sending the shuttle status information indicating the end of the shuttle to the first terminal, a user can control the first terminal using its input device.

[0023] In another possible implementation, the long press operation can include a press operation and a release operation; the second terminal receiving the long press operation input by the user using the input device of the first terminal can include: the second terminal receiving a fourth operation parameter from the first terminal, the fourth operation parameter being the operation parameter contained in the press event received by the first terminal after the user performs a press operation using the input device of the first terminal when the first cursor is displayed on the status bar; the second terminal receiving a fifth operation parameter from the first terminal, the fifth operation parameter being the operation parameter contained in the release event received by the first terminal after the user performs a press operation for a period of time and then performs a release operation using the input device of the first terminal; the second terminal simulating a press event according to the fourth operation parameter and simulating a release event according to the fifth operation parameter, the press event and the release event being used to trigger the application's interface windowing; in response to the long press operation, the second terminal displays a window, including: in response to the press event and the release event, the second terminal displays a window.

[0024] In another possible implementation, the method may further include: after the second terminal successfully establishes a connection with the first terminal, it creates a virtual input device; or, the second terminal receives a notification message from the first terminal indicating that the keyboard and mouse sharing mode of the first terminal has been enabled, and in response to the notification message, the second terminal creates a virtual input device; wherein the virtual input device is used by the second terminal to simulate input events according to operating parameters. By using keyboard and mouse sharing technology, dragging and dropping objects between multiple terminals using the input device of one terminal is realized.

[0025] A second aspect of this application provides a cross-device object dragging method, which can be applied to a first terminal connected to a second terminal. The method may include: the first terminal sending data of a dragging operation input by a user using an input device on the first terminal to the second terminal, the dragging operation being used to initiate dragging of an object displayed on the second terminal; the first terminal receiving drag data from the second terminal, the drag data being sent to the first terminal after the second terminal determines that the object has been dragged off the edge of the second terminal's display screen as the object moves with a first cursor on the second terminal's display screen.

[0026] As an example, the input device of the first terminal mentioned above can be a mouse, touchpad, etc. The first cursor can be a cursor displayed on the screen of the second terminal.

[0027] By adopting the above technical solution, in scenarios where multiple terminals are used collaboratively, users can use input devices such as a mouse to drag and drop objects from one terminal to another, following the cursor. For example, a user can use the input device of one terminal to drag and drop objects from one terminal to another, improving the efficiency of multi-terminal collaborative use, increasing the interaction between terminals, and enhancing the user experience.

[0028] In one possible implementation, after the first terminal receives drag data from the second terminal, the method may further include: the first terminal displaying an animation of an object moving with a second cursor on its screen, based on the drag data and the drag operation. The second cursor can be a cursor displayed on the first terminal's screen. Based on the received drag data and user input, the animation of the object moving with the cursor can continue to be displayed on the first terminal, thus achieving drag continuation.

[0029] In another possible implementation, the object can be an application icon; or, the object can be a window that includes the application's interface. When the object is an application icon, the drag data can include the application icon; when the object is a window, the drag data can include the application's interface.

[0030] In another possible implementation, after the first terminal receives drag data from the second terminal, the method further includes: the first terminal receiving a drag-release operation input by a user using an input device on the first terminal; in response to the drag-release operation, the first terminal displays the application's interface on its display screen. After the dragging of the object is completed, the first terminal may display the interface of the application corresponding to the dragged object on its display screen.

[0031] In another possible implementation, the first terminal displays the application interface on its screen, which may include: the first terminal sending an instruction message to the second terminal, the instruction message being used to indicate the end of dragging an object; the first terminal receiving data from the second terminal; and the first terminal displaying the application interface on its screen based on the data.

[0032] In another possible implementation, when the object is a window, before the first terminal sends the drag operation data entered by the user using the input device of the first terminal to the second terminal, the method may further include: the first terminal sending the long press operation data entered by the user using the input device of the first terminal to the second terminal, wherein the long press operation is entered by the user when the first cursor is displayed on the status bar of the second terminal, and is used to trigger the windowing of the application interface displayed on the second terminal; the window specifically includes a screenshot of the application interface, and the drag data specifically includes a screenshot of the application interface.

[0033] In another possible implementation, the first terminal displays an animation of an object moving with the second cursor on its display screen based on drag data and drag operation. This may include: the first terminal displays an animation of the object's shadow moving with the second cursor on its display screen based on drag data and drag operation.

[0034] In another possible implementation, the object can be text, a file, or a folder; the drag data includes a bitmap of the drag event content and the shadow; where, when the object is text, the drag event content includes the text, and when the object is a file or folder, the drag event content is the file path.

[0035] In another possible implementation, before the first terminal sends data about the drag operation input by the user using the input device of the first terminal to the second terminal, the method may further include: the first terminal determining that the second cursor has slipped off the edge of the display screen of the first terminal. The user triggers cursor movement by slipping the cursor on the display screen of the first terminal off the edge of the display screen.

[0036] In another possible implementation, before the first terminal determines that the second cursor has slid off the edge of the display screen of the first terminal, the method may further include: the first terminal receiving an operation by the user to move the second cursor using the input device of the first terminal; in response to the operation of moving the second cursor, the first terminal displaying an animation of the movement of the second cursor on the display screen of the first terminal.

[0037] In another possible implementation, after the first terminal determines that the second cursor has slid off the edge of the first terminal's display screen, the method may further include: the first terminal sending first shuttle status information to the second terminal, the first shuttle status information indicating the start of shuttle. After the cursor shuttles, by sending shuttle status information indicating the start of shuttle to other terminals, other terminals are prepared to continue dragging, such as by displaying the cursor.

[0038] In another possible implementation, after the first terminal determines that the second cursor has moved out of the edge of the first terminal's display screen, the method may further include: the first terminal sending data of the user's input operation on the first terminal using its input device to move the first cursor to the second terminal, for the second terminal to display an animation of the first cursor movement on the second terminal's display screen. After the cursor moves, the first terminal sends the user's input operation to other terminals to enable the first terminal's input device to control other terminals.

[0039] In another possible implementation, the drag operation may include a press operation and a move operation; the first terminal sends drag operation data input by the user using the input device of the first terminal to the second terminal, including: after the user performs a press operation using the input device of the first terminal, the first terminal intercepts the press event; the first terminal sends a first operation parameter included in the press event to the second terminal; after the user performs a move operation using the input device of the first terminal, the first terminal intercepts the move event; the first terminal sends a second operation parameter included in the move event to the second terminal; wherein the first operation parameter and the second operation parameter are used by the second terminal to simulate the press event and the move event, respectively, and then used to instruct the initiation of dragging of the target object.

[0040] In another possible implementation, the first terminal sends data of the user's operation of moving the first cursor using the input device of the first terminal to the second terminal, including: after the user performs the movement operation using the input device of the first terminal, the first terminal intercepts the movement event; the first terminal sends a third operation parameter including the movement event to the second terminal, the third operation parameter being used by the second terminal to simulate the movement event, and then used to instruct the movement of the first cursor.

[0041] In another possible implementation, before the first terminal displays an animation of an object moving with a second cursor on its display screen based on drag data and drag operation, the method may further include: the first terminal receiving second shuttle state information from the second terminal, the second shuttle state information being used to indicate the end of the shuttle.

[0042] In another possible implementation, before the first terminal receives drag data from the second terminal, the method may further include: the first terminal sending a request message to the second terminal, the request message being used to request drag data.

[0043] In another possible implementation, after the first terminal receives the second shuttle state information from the second terminal, the method may further include: the first terminal canceling the interception of the input event; the first terminal displaying an animation of the object moving with the second cursor on its display screen based on the drag operation and drag data, including: after the user performs a movement operation using the input device of the first terminal, the first terminal receives a movement event; the first terminal generates a press event; the first terminal displays an animation of the object moving with the second cursor on its display screen based on the movement event, the press event, and the drag data.

[0044] In another possible implementation, after the first terminal determines that the second cursor has slid off the edge of the first terminal's display screen, the method may further include: the first terminal displaying an invisible window whose transparency is greater than a threshold, the invisible window being used to initiate a drag event.

[0045] In another possible implementation, the long press operation includes a press operation and a release operation; the first terminal sends data of the long press operation input by the user using the input device of the first terminal to the second terminal, including: after the user performs a press operation using the input device of the first terminal, the first terminal intercepts the press event and sends a fourth operation parameter included in the press event to the second terminal; after the user performs a release operation using the input device of the first terminal for a period of time after performing the press operation, the first terminal intercepts the release event and sends a fifth operation parameter included in the release event to the second terminal; wherein, the fourth operation parameter and the fifth operation parameter are used by the second terminal to simulate the press event and the release event, respectively, and then to trigger the windowing of the application interface.

[0046] A third aspect of this application provides a cross-device object dragging device, which can be applied to a second terminal connected to a first terminal. The device may include: a display unit for displaying a first cursor on an object displayed on the second terminal; a receiving unit for receiving a drag operation input by a user using an input device of the first terminal, the drag operation being used to initiate dragging of an object; the display unit is further configured to display an animation of the object moving with the first cursor on the display screen of the second terminal in response to the drag operation; and a sending unit for sending drag data to the first terminal when it is determined that the object has been dragged out of the edge of the display screen of the second terminal.

[0047] In one possible implementation, dragging data can be used to display an animation of an object moving with a second cursor on the first terminal's screen.

[0048] In another possible implementation, the object can be an application icon; or, the object can be a window that includes the application's interface; when the object is an application icon, the dragged data includes the application icon; when the object is a window, the dragged data includes the application's interface.

[0049] In another possible implementation, the receiving unit is further configured to receive an indication message from the first terminal, which is sent by the first terminal after the user inputs a drag-and-release operation using the input device of the first terminal, and is used to indicate the end of dragging the object; the sending unit is further configured to send data to the first terminal, which is used by the first terminal to display the application interface on the display screen of the first terminal.

[0050] In another possible implementation, when the object is an application icon, the device further includes: a creation unit; a creation unit for creating a draggable object based on the application icon, the draggable object including: an icon identical to the application icon; and a display unit specifically for displaying an animation of the draggable object moving with a first cursor in response to a drag operation.

[0051] In another possible implementation, when the object is a window, the display unit is also used to display the application's interface; the receiving unit is also used to receive a long press operation input by the user using the input device of the first terminal when the first cursor is displayed on the status bar of the second terminal, which is used to trigger the application's interface windowing; the display unit is also used to display the window in response to the long press operation; the window specifically includes a screenshot of the application's interface, and the dragged data specifically includes a screenshot of the application's interface.

[0052] In another possible implementation, the display unit is specifically used to display an animation of the shadow of an object moving with the first cursor on the display screen of the second terminal.

[0053] In another possible implementation, the object can be text, a file, or a folder; the drag data includes a bitmap of the drag event content and the shadow; where, when the object is text, the drag event content includes the text, and when the object is a file or folder, the drag event content is the file path.

[0054] In another possible implementation, the device may also include an acquisition unit for retrieving and caching dragged data.

[0055] In another possible implementation, the receiving unit is also used to receive a request message from the first terminal, the request message being used to request drag data; the sending unit is specifically used to send the cached drag data to the first terminal.

[0056] In another possible implementation, the receiving unit is further configured to receive first shuttle status information from the first terminal, the first shuttle status information being used to indicate the start of shuttle; the display unit is further configured to display a first cursor on the second terminal in response to the first shuttle status information.

[0057] In another possible implementation, the receiving unit is further configured to receive an operation of moving the first cursor input by the user using the input device of the first terminal; the display unit is further configured to, in response to the operation of moving the first cursor, display an animation of the movement of the first cursor on the display screen of the second terminal until the first cursor moves onto the object.

[0058] In another possible implementation, the drag operation includes a press operation and a move operation; a receiving unit is specifically used to receive a first operation parameter from the first terminal, the first operation parameter being the operation parameter contained in a press event received by the first terminal after the user performs a press operation using the input device of the first terminal; and to receive a second operation parameter from the first terminal, the second operation parameter being the operation parameter contained in a move event received by the first terminal after the user performs a move operation using the input device of the first terminal; the device may further include: a simulation unit, used to simulate a press event according to the first operation parameter, and to simulate a move event according to the second operation parameter, the press event and the move event being used to indicate the initiation of dragging an object; and a display unit, specifically used to display an animation of the object moving with the first cursor in response to the press event and the move event.

[0059] In another possible implementation, the receiving unit is specifically used to receive a third operation parameter from the first terminal. The third operation parameter is the operation parameter contained in the movement event received by the first terminal after the user performs a movement operation using the input device of the first terminal. The device may also include: a simulation unit, used to simulate a movement event according to the third operation parameter, the movement event being used to indicate the movement of the first cursor.

[0060] In another possible implementation, the sending unit is also used to send second shuttle status information to the first terminal, the second shuttle status information being used to indicate the end of the shuttle.

[0061] In another possible implementation, the long press operation includes a press operation and a release operation; a receiving unit is specifically used to receive a fourth operation parameter from the first terminal, the fourth operation parameter being the operation parameter contained in the press event received by the first terminal after the user performs a press operation using the input device of the first terminal when the first cursor is displayed on the status bar; and to receive a fifth operation parameter from the first terminal, the fifth operation parameter being the operation parameter contained in the release event received by the first terminal after the user performs a press operation for a period of time and then performs a release operation using the input device of the first terminal; the device may further include: a simulation unit, used to simulate a press event according to the fourth operation parameter, and a release event according to the fifth operation parameter, the press event and the release event being used to trigger the application's interface windowing; and a display unit, specifically used to display a window in response to the press event and the release event.

[0062] In another possible implementation, the creation unit is further configured to create a virtual input device after the connection with the first terminal is successfully established; or, the receiving unit is further configured to receive a notification message from the first terminal, the notification message indicating that the keyboard and mouse sharing mode of the first terminal has been enabled, and the creation unit is further configured to create a virtual input device in response to the notification message; wherein, the virtual input device is used by the second terminal to simulate input events according to operating parameters.

[0063] A fourth aspect of this application provides a cross-device object dragging device, which is applied to a first terminal and connected to a second terminal. The device may include: a sending unit for sending data of a dragging operation input by a user using an input device of the first terminal to the second terminal, wherein the dragging operation is used to initiate dragging of an object displayed on the second terminal; and a receiving unit for receiving dragging data from the second terminal, wherein the dragging data is sent to the first terminal after the second terminal determines that the object has been dragged out of the edge of the display screen of the second terminal during the process of the object moving on the display screen of the second terminal with the first cursor.

[0064] In one possible implementation, the device may further include a display unit for displaying an animation of an object moving with a second cursor on a display screen of a first terminal, based on drag data and drag operations.

[0065] In another possible implementation, the object can be an application icon; or, the object can be a window that includes the application's interface; when the object is an application icon, the dragged data includes the application icon; when the object is a window, the dragged data includes the application's interface.

[0066] In another possible implementation, the receiving unit is further configured to receive a drag-and-release operation input by the user using the input device of the first terminal; the display unit is further configured to display the application interface on the display screen of the first terminal in response to the drag-and-release operation.

[0067] In another possible implementation, the sending unit is further configured to send an instruction message to the second terminal, the instruction message indicating the end of dragging the object; the receiving unit is further configured to receive data from the second terminal; and the display unit is specifically configured to display the application interface on the display screen of the first terminal according to the data.

[0068] In another possible implementation, when the object is a window, the sending unit is also used to send data of a long press operation input by the user using the input device of the first terminal to the second terminal. The long press operation is input by the user when the first cursor is displayed on the status bar of the second terminal, and is used to trigger the windowing of the application interface displayed on the second terminal. The window specifically includes a screenshot of the application interface, and the dragged data specifically includes a screenshot of the application interface.

[0069] In another possible implementation, the display unit is specifically used to display an animation of the shadow of an object moving with a second cursor on the display screen of the first terminal, based on the drag data and drag operation.

[0070] In another possible implementation, the object can be text, a file, or a folder; the drag data includes a bitmap of the drag event content and the shadow; where, when the object is text, the drag event content includes the text, and when the object is a file or folder, the drag event content is the file path.

[0071] In another possible implementation, the device may further include a determining unit for determining the edge of the second cursor sliding off the display screen of the first terminal.

[0072] In another possible implementation, the receiving unit is further configured to receive an operation of moving the second cursor input by the user using the input device of the first terminal; the display unit is further configured to display an animation of the movement of the second cursor on the display screen of the first terminal in response to the operation of moving the second cursor.

[0073] In another possible implementation, the sending unit is also used to send first shuttle status information to the second terminal, the first shuttle status information being used to indicate the start of shuttle.

[0074] In another possible implementation, the sending unit is further configured to send data of the operation of moving the first cursor input by the user using the input device of the first terminal to the second terminal, so that the second terminal can display the animation of the first cursor movement on the display screen of the second terminal.

[0075] In another possible implementation, the drag operation includes a press operation and a move operation; the device may further include: an interception unit; the interception unit is configured to intercept a press event after the user performs a press operation using the input device of the first terminal; a sending unit is specifically configured to send a first operation parameter including the press event to the second terminal; the interception unit is further configured to intercept a move event after the user performs a move operation using the input device of the first terminal; the sending unit is specifically configured to send a second operation parameter including the move event to the second terminal; wherein the first operation parameter and the second operation parameter are respectively used by the second terminal to simulate the press event and the move event, and then to indicate the initiation of dragging of an object.

[0076] In another possible implementation, the interception unit is used to intercept the movement event after the user performs a movement operation using the input device of the first terminal; the sending unit is specifically used to send the third operation parameters included in the movement event to the second terminal, the third operation parameters being used by the second terminal to simulate the movement event, and then to instruct the movement of the first cursor.

[0077] In another possible implementation, the receiving unit is also configured to receive second shuttle status information from the second terminal, the second shuttle status information being used to indicate the end of the shuttle.

[0078] In another possible implementation, the sending unit is also used to send a request message to the second terminal, the request message being used to request drag data.

[0079] In another possible implementation, the device may further include: an interception unit and an input unit; the interception unit is used to cancel the interception of the input event; the input unit is used to receive the movement event after the user performs a movement operation using the input device of the first terminal; the input unit is also used to generate a press event; and the display unit is specifically used to display an animation of the object moving with the second cursor on the display screen of the first terminal according to the movement event, the press event and the drag data.

[0080] In another possible implementation, the display unit is also used to display an invisible window with an opacity greater than a threshold. The invisible window is used to initiate drag events.

[0081] In another possible implementation, the long press operation includes a press operation and a release operation; an interception unit is used to intercept the press event after the user performs the press operation using the input device of the first terminal; a sending unit is specifically used to send the fourth operation parameter included in the press event to the second terminal; the interception unit is also used to intercept the release event after the user performs the release operation using the input device of the first terminal for a period of time after performing the press operation; the sending unit is also used to send the fifth operation parameter included in the release event to the second terminal; wherein, the fourth operation parameter and the fifth operation parameter are used by the second terminal to simulate the press event and the release event, respectively, and then to trigger the windowing of the application interface.

[0082] A fifth aspect of this application provides a cross-device object dragging device, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement, when executing the instructions, a method as described in any one of the first aspect or any possible implementation thereof, or a method as described in any one of the second aspect or any possible implementation thereof.

[0083] A sixth aspect of this application provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the method as described in any one of the first aspect or any possible implementation thereof, or implement the method as described in any one of the second aspect or any possible implementation thereof.

[0084] A seventh aspect of this application provides a terminal including a display screen, one or more processors, and a memory; the display screen, processors, and memory are coupled; the memory is used to store computer program code, the computer program code including computer instructions, which, when executed by the terminal, cause the terminal to perform a method as described in any one of the first aspect or any possible implementation thereof, or cause the terminal to perform a method as described in any one of the second aspect or any possible implementation thereof.

[0085] An eighth aspect of this application provides a computer program product comprising computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is executed in a terminal, a processor in the terminal executes the method of any one of the first aspect or any possible implementation thereof, or executes the method of any one of the second aspect or any possible implementation thereof.

[0086] A ninth aspect of this application provides a cross-device object drag-and-drop system, which may include a first terminal and a second terminal, the first terminal being connected to the second terminal.

[0087] The second terminal displays a first cursor on the object displayed on the second terminal; the first terminal sends drag operation data input by the user using the input device of the first terminal to the second terminal; the second terminal receives the drag operation input by the user using the input device of the first terminal, which is used to initiate a drag on an object; in response to the drag operation, the second terminal displays an animation of the object moving with the first cursor on the display screen of the second terminal; when the second terminal determines that the object has been dragged off the edge of the display screen of the second terminal, it sends drag data to the first terminal. The first terminal receives the drag data from the second terminal.

[0088] In one possible implementation, the first terminal displays an animation of an object moving with a second cursor on its screen, based on the drag data and drag operation.

[0089] In another possible implementation, after determining that the second cursor has slid off the edge of the display screen of the first terminal, the first terminal sends data of the drag operation input by the user using the input device of the first terminal to the second terminal.

[0090] In another possible implementation, after the first terminal determines that the second cursor has slid out of the edge of the first terminal's display screen, the first terminal sends first shuttle status information to the second terminal, which is used to indicate that the shuttle has started.

[0091] In another possible implementation, the second terminal is also used to send second shuttle status information to the first terminal after determining that the object has been dragged off the edge of the second terminal's display screen. The second shuttle status information is used to indicate the end of the shuttle.

[0092] It should be noted that, in this embodiment, the object being dragged out of the edge of the terminal (such as the second terminal) display screen can mean that a part of the object is dragged out (or overflows) the terminal display screen, or that the entire area of ​​the object is dragged out (or overflows) the terminal display screen, or that the cursor slides out of the edge of the terminal display screen when the object moves on the terminal display screen. This embodiment does not impose any specific limitations here.

[0093] It is understood that the beneficial effects achieved by the cross-device object drag-and-drop device described in the third aspect and any possible implementation thereof, the cross-device object drag-and-drop device described in the fourth aspect and any possible implementation thereof, the cross-device object drag-and-drop device described in the fifth aspect, the computer-readable storage medium described in the sixth aspect, the terminal described in the seventh aspect, the computer program product described in the eighth aspect, and the cross-device object drag-and-drop system described in the ninth aspect can be referred to as the beneficial effects described in the first or second aspect and any possible implementation thereof, which will not be repeated here. Attached Figure Description

[0094] Figure 1 A simplified schematic diagram of a system architecture provided for an embodiment of this application;

[0095] Figure 2A This is a schematic diagram of the structure of a mobile phone provided in an embodiment of this application;

[0096] Figure 2B A schematic diagram illustrating the composition of a software architecture provided in an embodiment of this application;

[0097] Figure 3 A flowchart illustrating a cross-device object dragging method provided in an embodiment of this application;

[0098] Figure 4 A schematic diagram of a coordinate system on a display screen provided in an embodiment of this application;

[0099] Figure 5A A schematic diagram of a cross-device object drag-and-drop interface provided in an embodiment of this application;

[0100] Figure 5B This is a schematic diagram of another cross-device object drag-and-drop interface provided in an embodiment of this application;

[0101] Figure 6 This is a schematic diagram of another cross-device object drag-and-drop interface provided in an embodiment of this application;

[0102] Figure 7 This is a schematic diagram of another cross-device object drag-and-drop interface provided in an embodiment of this application;

[0103] Figure 8 This is a schematic diagram of another cross-device object drag-and-drop interface provided in an embodiment of this application;

[0104] Figure 9 A flowchart illustrating another cross-device object dragging method provided in an embodiment of this application;

[0105] Figure 10 This is a schematic diagram of another cross-device object drag-and-drop interface provided in an embodiment of this application;

[0106] Figure 11 This is a schematic diagram of another cross-device object drag-and-drop interface provided in an embodiment of this application;

[0107] Figure 12 This is a schematic diagram of another cross-device object drag-and-drop interface provided in an embodiment of this application;

[0108] Figure 13 This is a schematic diagram of another cross-device object drag-and-drop interface provided in an embodiment of this application;

[0109] Figure 14 This is a schematic diagram of another cross-device object drag-and-drop interface provided in an embodiment of this application;

[0110] Figure 15 A flowchart illustrating yet another cross-device object dragging method provided in an embodiment of this application;

[0111] Figure 16 This is a schematic diagram of another cross-device object drag-and-drop interface provided in an embodiment of this application;

[0112] Figure 17 A schematic diagram of a data structure for a drag-and-drop event on a Windows device, provided as an embodiment of this application;

[0113] Figure 18 This is a schematic diagram of a data structure for drag-and-drop events on an Android device, provided in an embodiment of this application.

[0114] Figure 19 This is a schematic diagram of another cross-device object drag-and-drop interface provided in an embodiment of this application;

[0115] Figure 20 A flowchart illustrating yet another cross-device object dragging method provided in an embodiment of this application;

[0116] Figure 21 This is a schematic diagram of another cross-device object drag-and-drop interface provided in an embodiment of this application;

[0117] Figure 22 This is a schematic diagram of another cross-device object drag-and-drop interface provided in an embodiment of this application;

[0118] Figure 23 A schematic diagram of the composition of a cross-device object dragging device provided in an embodiment of this application;

[0119] Figure 24 A schematic diagram illustrating the composition of another cross-device object dragging device provided in an embodiment of this application;

[0120] Figure 25 This is a schematic diagram of the composition of a chip system provided in an embodiment of this application. Detailed Implementation

[0121] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of this application, unless otherwise stated, "a plurality of" means two or more.

[0122] This application provides a cross-device object drag-and-drop method and device, which can be applied to scenarios where multiple terminals are used collaboratively. In this scenario, using keyboard and mouse sharing technology, a single input device (such as a mouse or touchpad) can be used to control all terminals. For example, objects such as application icons or application interfaces can be dragged and dropped across devices. Furthermore, text or files (or objects) can be transferred between multiple terminals participating in the collaborative use through drag-and-drop, allowing users to process the transferred content using these terminals. The method provided in this embodiment increases the interaction between terminals during multi-terminal collaborative use, improving the user experience.

[0123] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0124] Please refer to Figure 1 This is a simplified schematic diagram illustrating a system architecture to which the above-described method can be applied, provided in an embodiment of this application. This system architecture can be the cross-device object drag-and-drop system described in this embodiment. Figure 1 As shown, the system architecture may include at least: a first terminal 101 and a second terminal 102.

[0125] The first terminal 101 is connected to the input device 101-1 (e.g., Figure 1 As shown), or including input device 101-1 (as shown). Figure 1(Not shown in the image). As an example, the input device 101-1 can be a mouse, touchpad, touchscreen, etc. Figure 1 The example shown is a mouse, which is the input device 101-1.

[0126] In this embodiment, the first terminal 101 and the second terminal 102 can establish a connection via wired or wireless means. Based on the established connection, the first terminal 101 and the second terminal 102 can be used together. In this embodiment, when the first terminal 101 and the second terminal 102 establish a wireless connection, the wireless communication protocol used can be Wi-Fi, Bluetooth, ZigBee, Near Field Communication (NFC), or various cellular network protocols, without specific limitations.

[0127] After the first terminal 101 and the second terminal 102 are connected, using keyboard and mouse sharing technology, the user can use a single input device, such as the aforementioned input device 101-1, to control both the first terminal 101 and the second terminal 102. In other words, the user can not only use the input device 101-1 of the first terminal 101 to control the first terminal 101, but the first terminal 101 can also share its input device 101-1 with the second terminal 102, allowing the user to control the second terminal 102.

[0128] For example, one or more applications are installed on the second terminal 102. The second terminal 102 can display the icons of the corresponding applications on its display screen 102-1. In this embodiment, after the first terminal 101 and the second terminal 102 establish a connection, using keyboard and mouse sharing technology, the user can use the aforementioned input device 101-1 to drag and drop the application icons from the display screen 102-1 of the second terminal 102 to the display screen 101-2 of the first terminal 101.

[0129] For example, the application interface is displayed on the screen 102-1 of the second terminal 102. In this embodiment, after the first terminal 101 and the second terminal 102 establish a connection, the user can use the above-mentioned input device 101-1 to drag the application interface from the screen 102-1 of the second terminal 102 to the screen 101-2 of the first terminal 101 using keyboard and mouse sharing technology.

[0130] For example, taking the input device 101-1 mentioned above as a mouse, after the first terminal 101 is connected to the second terminal 102, without starting screen mirroring, using keyboard and mouse sharing technology, the user can use the mouse to drag and drop text or files from the first terminal 101 to the second terminal 102. The user can also use the mouse to drag and drop text or files from the second terminal 102 to the first terminal 101.

[0131] It should be noted that, without enabling screen mirroring, this embodiment of the application can not only achieve drag-and-drop of content between two terminals, but also between three or more terminals. For example, when the second terminal 102 is connected to other devices, such as a third terminal, in this embodiment, without enabling screen mirroring, using keyboard and mouse sharing technology, after dragging text or files from one terminal to another, the user can continue to drag the content to a third terminal. For instance, continuing with the example of a mouse as the input device 101-1, without enabling screen mirroring, the user can use the mouse to drag text or files from the first terminal 101 to the second terminal 102, and then continue to drag the content to the third terminal. The drag-and-drop is complete after the user releases the mouse.

[0132] It should be noted that the terminals in this application embodiment, such as the first terminal 101, the second terminal 102, and the third terminal, can be mobile phones, tablets, handheld computers, PCs, cellular phones, personal digital assistants (PDAs), wearable devices (such as smartwatches), smart home devices (such as televisions), in-vehicle computers, game consoles, and augmented reality (AR) / virtual reality (VR) devices, etc. This embodiment does not impose any special restrictions on the specific form of the terminal. Figure 1 The example shown uses the first terminal 101 as a PC and the second terminal 102 as a mobile phone.

[0133] In this embodiment, a mobile phone is used as the terminal. Please refer to... Figure 2A This is a schematic diagram of the structure of a mobile phone provided in an embodiment of this application.

[0134] like Figure 2AAs shown, a mobile phone may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0135] The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0136] It is understood that the structure illustrated in this embodiment does not constitute a specific limitation on the mobile phone. In other embodiments, the mobile phone may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0137] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0138] The controller can be the nerve center and command center of a mobile phone. Based on the instruction opcode and timing signals, the controller generates operation control signals to control the fetching and execution of instructions.

[0139] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0140] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM interface, and / or a USB interface, etc.

[0141] The charging management module 140 receives charging input from the charger. While charging the battery 142, the charging management module 140 can also power the mobile phone via the power management module 141. The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 can also receive input from the battery 142 to power the mobile phone.

[0142] The wireless communication function of a mobile phone can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor, and baseband processor.

[0143] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the mobile phone can be used to cover one or more communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0144] The mobile communication module 150 can provide solutions for wireless communication applications in mobile phones, including 2G / 3G / 4G / 5G. The mobile communication module 150 may include at least one filter, switch, power amplifier, low-noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0145] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0146] The wireless communication module 160 can provide solutions for wireless communication applications in mobile phones, including wireless local area networks (WLANs) (such as Wi-Fi), Bluetooth, Global Navigation Satellite System (GNSS), frequency modulation (FM), NFC, and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0147] In some embodiments, antenna 1 of the mobile phone is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling the mobile phone to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time-Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. The GNSS may include Global Positioning System (GPS), Global Navigation Satellite System (GLONASS), BeiDou Navigation Satellite System (BDS), Quasi-Zenith Satellite System (QZSS), and / or Satellite Based Augmentation Systems (SBAS).

[0148] The mobile phone implements its display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0149] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the mobile phone may include one or N displays 194, where N is a positive integer greater than 1.

[0150] The mobile phone can perform shooting functions through an ISP, camera 193, video codec, GPU, display 194, and application processor. In some embodiments, the mobile phone may include one or N cameras 193, where N is a positive integer greater than 1.

[0151] The external storage interface 120 can be used to connect an external storage card, such as a Micro SD card, to expand the phone's storage capacity. The external storage card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external storage card.

[0152] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various mobile phone functions and data processing by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during mobile phone use (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0153] Mobile phones can perform audio functions, such as music playback and recording, through components like the audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0154] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A may be disposed on display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. When a touch operation is applied to display screen 194, the mobile phone detects the intensity of the touch operation based on pressure sensor 180A. The mobile phone can also calculate the touch location based on the detection signal from pressure sensor 180A.

[0155] The gyroscope sensor 180B is used to determine the phone's motion posture. The barometric pressure sensor 180C is used to measure air pressure. The magnetic sensor 180D includes a Hall effect sensor. The phone can use the magnetic sensor 180D to detect the opening and closing of the flip case. The accelerometer sensor 180E can detect the magnitude of the phone's acceleration in various directions (generally three axes). The proximity sensor 180F is used to measure distance. The phone can use the proximity sensor 180G to detect when the user holds the phone close to their ear for a call, automatically turning off the screen to save power. The proximity sensor 180G can also be used for automatic unlocking and locking in flip case and pocket modes. The ambient light sensor 180L is used to sense ambient light brightness. The fingerprint sensor 180H is used to collect fingerprints. The phone can use the collected fingerprint characteristics to achieve fingerprint unlocking, app access lock, fingerprint photography, fingerprint answering of calls, etc. The temperature sensor 180J is used to detect temperature.

[0156] Touch sensor 180K, also known as a "touch panel," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touchscreen." Touch sensor 180K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of the phone, in a different position than display screen 194.

[0157] The bone conduction sensor 180M can acquire vibration signals. Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch buttons. The motor 191 can generate vibration alerts. The motor 191 can be used for incoming call vibration alerts or for touch vibration feedback. The indicator 192 can be an indicator light, used to indicate charging status, battery level changes, or to indicate messages, missed calls, notifications, etc.

[0158] The SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to achieve contact and separation with the mobile phone. The mobile phone can support one or N SIM card interfaces, where N is a positive integer greater than 1. The mobile phone interacts with the network through the SIM card to achieve functions such as calls and data communication. In some embodiments, the mobile phone uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the mobile phone and cannot be separated from it. The methods in the following embodiments can be implemented in a mobile phone with the above hardware structure.

[0159] Combination Figure 1 This application embodiment uses the example of a first terminal 101 having a Windows system and a second terminal 102 having an Android system to illustrate the software structure of the first terminal 101 and the second terminal 102. Please refer to... Figure 2B This is a schematic diagram of a software architecture provided in an embodiment of this application.

[0160] Among them, such as Figure 2B As shown, the software architecture of the first terminal 101 may include an application layer and a Windows system (Windows shell). In some embodiments, the application layer may include various applications installed on the first terminal 101. The applications in the application layer can interact directly with the Windows system. For example, the application layer may also include a keyboard and mouse module, a drag-and-drop management module, and a transfer management module. Further, the application layer may also include a window management module.

[0161] The software system of the second terminal 102 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. Taking a layered architecture as an example, the software system of the second terminal 102 is divided into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, such as... Figure 2BAs shown, the second terminal 102 may include an application layer and a framework layer (FWK). The application layer may include a series of application packages. For example, the application packages may include applications such as settings, calculator, camera, SMS, and music player. The applications included in the application layer may be system applications of the second terminal 102 or third-party applications; this embodiment does not impose specific limitations here. The application layer may also include a transmission management module and a drag-and-drop management module. The application layer may also include a launcher. The application layer may also include a system interface (systemUI). The framework layer is mainly responsible for providing application programming interfaces (APIs) and programming frameworks for the applications in the application layer. In some embodiments, the framework layer may include a window manager (or window management service). Of course, the second terminal 102 may also include other layers, such as the kernel layer ( Figure 2B (Not shown in the image). This kernel layer is the layer between hardware and software. The kernel layer can at least contain display drivers, camera drivers, audio drivers, sensor drivers, etc.

[0162] In scenarios involving the collaborative use of multiple terminals, to improve efficiency, users can control these multiple terminals using a single input device. In this embodiment, combined with... Figure 1 Taking multiple terminals, including a first terminal 101 and a second terminal 102, and using a mouse as the input device 101-1 of the first terminal 101 as an example, after the first terminal 101 and the second terminal 102 establish a connection, based on the above software architecture and with the help of keyboard and mouse sharing technology, the user can use the input device 101-1 of the first terminal 101 to control the first terminal 101 and the second terminal 102. For example, in some embodiments, the user can use the input device 101-1 of the first terminal 101 to drag and drop the icon or interface of an application displayed on the second terminal 102 from the second terminal 102 to the first terminal 101. As another example, in some embodiments, without initiating screen mirroring, the user can use the input device 201-1 of the first terminal 101 to transfer files or text content from the first terminal 201 to the second terminal 202 by dragging and dropping. Similarly, files or text content from the second terminal 202 can also be transferred to the first terminal 201 by dragging and dropping. In other words, without enabling screen mirroring, users can use the input device 201-1 of the first terminal 201 to drag and drop files or text content from the application between the first terminal 201 and the second terminal 202.

[0163] Among them, keyboard and mouse sharing technology can refer to the technology of using the input device (such as mouse or touchpad) of one terminal to control other terminals.

[0164] It should be noted that in this embodiment, the drag-and-drop management module can also be called a drag-and-drop service module. Furthermore, in the above embodiment, it can be seen that both the first terminal 101 and the second terminal 102 include a transmission management module, and communication between the first terminal 101 and the second terminal 102 can be achieved through the transmission management module. In some other embodiments, the drag-and-drop management module may also have the function of communicating with other terminals; that is, neither the first terminal 101 nor the second terminal 102 may include a transmission management module, and communication between them may be achieved through the drag-and-drop management module. This embodiment does not impose specific limitations here. For ease of description, the following embodiments use the example of communication between the first terminal 101 and the second terminal 102 being achieved through a transmission management module for illustration.

[0165] In some embodiments, the following combination Figure 1 and Figure 2B Taking the first terminal 101 as a PC, the second terminal 102 as a mobile phone, and the input device 101-1 as a mouse as an example, the method provided in this embodiment will be described in detail with reference to the accompanying drawings.

[0166] Figure 3 This is a flowchart illustrating a cross-device object drag-and-drop method provided in an embodiment of this application. In this embodiment, the process of a user using a shared mouse on a PC to drag an application icon (the application icon being the dragged object) displayed on a mobile phone from the mobile phone to the PC is described. Figure 3 As shown, the method may include the following S301-S313.

[0167] S301. The mobile phone establishes a connection with the PC.

[0168] In some embodiments, the mobile phone and PC can establish a wired connection. For example, the mobile phone and PC can establish a wired connection via a data cable.

[0169] In other embodiments, the mobile phone and PC can establish a connection wirelessly. Establishing a wireless connection between terminals requires two things: first, the terminals must know each other's connection information; second, each terminal must have transmission capabilities. Connection information can be the terminal's device identifier, such as an Internet Protocol (IP) address, port number, or terminal login account. The terminal login account can be an account provided by the operator, such as a Huawei account. The terminal login account can also be an application account, such as a WeChat account or a Youku account. The terminal's transmission capability can be near-field communication (NFC) or long-range communication (LRC). That is, the wireless communication protocol used between terminals, such as a mobile phone and PC, to establish a connection can be a near-field communication protocol such as Wi-Fi, Bluetooth, or NFC, or a cellular protocol. Taking the wireless connection between a mobile phone and PC as an example, a user can touch the PC's NFC tag with their mobile phone. The mobile phone reads the connection information stored in the NFC tag, such as the PC's IP address. Then, the mobile phone can establish a connection with the PC using the NFC protocol based on the PC's IP address. For example, both the phone and PC have Bluetooth and Wi-Fi enabled. The PC can broadcast Bluetooth signals to discover nearby devices, such as displaying a list of discovered devices, which may include the phone's identifier. During device discovery, the PC can also exchange connection information, such as IP addresses, with the discovered devices. Then, after receiving a user's selection of the phone's identifier from the displayed device list, the PC can establish a connection with the phone using the Wi-Fi protocol based on the phone's IP address. Another example is when both the phone and PC are connected to a cellular network and logged into the same Huawei account. The phone and PC can establish a connection based on this Huawei account via the cellular network.

[0170] Once the mobile phone and PC successfully establish a connection, they can be used together. To improve the efficiency of this collaborative use, users can use a single input device, such as a PC mouse, to control both the PC and the mobile phone.

[0171] As an example, with the keyboard and mouse sharing mode enabled on the PC, a single input device can be used to control both the PC and the mobile phone.

[0172] For example, in some embodiments, after another terminal successfully establishes a connection with the PC, the PC may display a pop-up window asking the user whether to enable keyboard and mouse sharing mode. If the PC receives the user's selection to enable keyboard and mouse sharing mode, it can then enable the sharing mode.

[0173] After enabling keyboard and mouse sharing mode, a PC can notify all connected terminals that keyboard and mouse sharing mode is enabled. For example, if the PC is connected to a mobile phone, the PC will notify the mobile phone that keyboard and mouse sharing mode is enabled. Upon receiving this notification (e.g., a notification message), the mobile phone can create a virtual input device. This virtual input device functions the same as a regular input device such as a mouse or touchpad, and can be used by the mobile phone to simulate corresponding input events. For instance, taking a mouse as the input device, the virtual input device created by the mobile phone functions the same as a regular mouse; it can be considered a mouse shared by the PC to the mobile phone, and can be used to simulate mouse events on the mobile phone to achieve control of the mobile phone by the PC's mouse. For example, taking the mobile phone's operating system as Android, the mobile phone can utilize the Linux uinput capability to create a virtual input device. uinput is a kernel-level module that can simulate input devices. By writing to the / dev / uinput (or / dev / input / uinput) device, a process can create a virtual input device with specific functions. Once the virtual input device is created, it can simulate corresponding events. Similarly, other terminals connected to the PC will also create virtual input devices upon receiving the notification message. It should be noted that if the terminal receiving the notification message is running Android, the virtual input device can be created using Linux's uInput capability, or the Human Interface Device (HID) protocol can be used. If the terminal receiving the notification message is running iOS, Windows, or other operating systems, the HID protocol can be used. Furthermore, the above embodiment illustrates the creation of a virtual input device after a terminal connected to the PC receives a notification message indicating that keyboard and mouse sharing mode is enabled. In other embodiments, after receiving the notification message, the terminal connected to the PC may display a pop-up window asking the user whether they want to use the PC's input device to control the device. If the user selects to use the PC's input device, the virtual input device is created; otherwise, it is not created.

[0174] For example, in some other embodiments, after a mobile phone establishes a connection with a PC, the PC automatically enables keyboard and mouse sharing mode without the user having to manually enable it. On other terminals, such as the aforementioned mobile phone and PC, after establishing a connection, a virtual input device can also be created automatically without the PC sending a notification message. Alternatively, after another terminal establishes a connection with the PC, a pop-up window can first display asking the user if they want to use the PC's input device to control this device. If the user selects to use the PC's input device to control this device, then the virtual input device is automatically created; otherwise, it is not created.

[0175] In addition, combined Figure 1 Since the mouse is an input device for the PC, after other terminals, such as mobile phones, establish a connection with the PC, the PC typically temporarily responds to mouse operations; in other words, the user can temporarily control the PC using the mouse. In this embodiment, after the PC enables keyboard and mouse sharing mode, it can also trigger the other terminal, such as a mobile phone, that has established a connection with the PC and created a virtual input device to respond to mouse operations when the mouse travel conditions are met. That is, after the mouse travel conditions are met, the user can use the mouse to control other terminals that have established a connection with the PC and created a virtual input device, such as a mobile phone.

[0176] For example, a mouse hovering condition could be that the mouse pointer displayed on the PC screen moves across the edge of the PC screen. In other words, a user can move the mouse so that the mouse pointer displayed on the PC screen moves across the edge of the PC screen, triggering a response from another terminal that has established a connection with the PC and created a virtual input device to the mouse operation.

[0177] As an example implementation, after enabling keyboard and mouse sharing mode, the PC can enable input monitoring (monitor or detect) and attach a hook. Input monitoring can be used to monitor the relative displacement and coordinate position of the mouse pointer. The attached hook can be used to intercept (or block) the corresponding input event after mouse hovering begins. For example, if the input device is a mouse, the input event can be a mouse event (or a mouse event can be blocked), so that the mouse event is not transmitted to the PC's Windows system after being received by the PC's keyboard and mouse module. The attached hook can also be used to capture the intercepted input event, such as the parameters in the mouse event, after mouse hovering begins. The PC can use input monitoring to monitor the relative displacement and coordinate position of the mouse pointer and determine whether the mouse hovering conditions are met based on the monitored data. After determining that the mouse hovering conditions are met, the attached hook intercepts the mouse event, captures the operation parameters in the mouse event, and sends the captured operation parameters to other terminals connected to the PC that have created virtual input devices. This allows the terminals to use the created virtual input devices to simulate the corresponding input event, such as a mouse event, and then respond to it. This means that mouse operations can be handled by other terminals that have created virtual input devices connected to the PC.

[0178] Of course, input event interception and capture of operation parameters can also be achieved through other methods (such as registering RAWINPUT on the PC). Alternatively, input event interception and capture of operation parameters can be achieved separately in different ways. For example, taking a mouse as the input device, after the PC enables keyboard and mouse sharing mode, a hook can be mounted and RAWINPUT can be registered. After mouse hovering begins, the mounted hook can be used to intercept mouse events (or block mouse events), and the registered RAWINPUT can be used to capture the operation parameters in the intercepted mouse events. This embodiment does not limit the specific implementation of mouse event interception and capture of operation parameters. For ease of description, the following embodiment uses the example of intercepting input events and capturing operation parameters by mounting a hook.

[0179] For example, consider a user wanting to drag an application icon from their phone to their PC. The process that triggers the response from another terminal connected to the PC that has created a virtual input device, such as a mobile phone, to mouse operations may include the following steps S302-S304.

[0180] S302, The PC receives mouse movement events and displays an animation of mouse pointer movement on the PC's screen based on the mouse movement events.

[0181] It should be noted that the mouse pointer described in this embodiment can also be called a cursor. The cursor can be an image, and it can be dynamic or static; the cursor's style may also differ in different situations. This embodiment uses a mouse pointer as an example for explanation. The mouse pointer displayed on the PC screen can be the second cursor in this application.

[0182] S303, PC monitors the coordinates of the mouse pointer on the PC screen.

[0183] S304: Based on the coordinates of the mouse pointer on the PC screen, when the mouse pointer moves out of the edge of the PC screen, the PC intercepts the mouse movement event and sends the mouse operation parameter 1 contained in the mouse movement event to the mobile phone.

[0184] In this embodiment, after the keyboard and mouse sharing mode is enabled, when a user wants to use the mouse to control other terminals that have created virtual input devices connected to the PC, such as a mobile phone, such as wanting to drag the application icon displayed on the mobile phone from the mobile phone to the PC, the user can continuously move the mouse in the same direction, so that the mouse pointer displayed on the PC screen slides across (or slides out) the edge of the PC screen, thus triggering the mouse crossing condition.

[0185] For example, the PC can determine the coordinate position of the mouse pointer on the PC display screen based on the initial position and relative displacement of the mouse pointer, thereby determining whether the mouse pointer has slipped off the edge of the PC display screen.

[0186] The initial position of the mouse pointer can be the coordinates of the mouse pointer on the PC screen when the mouse begins to move, or the coordinates of the mouse pointer on the PC screen before the mouse begins to move. Specifically, this initial position can refer to the coordinates in a coordinate system with the top-left corner of the PC screen as the origin, the X-axis pointing from the top-left corner to the right edge of the PC screen, and the Y-axis pointing from the top-left corner to the bottom edge of the PC screen. For example, the specific process by which the PC determines whether the mouse pointer has slipped off the edge of the PC screen can be: combining... Figure 4 PC can establish a coordinate system with this initial coordinate position as the origin (e.g., Figure 4 The coordinate system shown (position o) has an X-axis pointing from the origin o to the right edge of the PC display screen, and a Y-axis pointing from the origin o to the top edge of the PC display screen. The PC can determine the coordinate values ​​of each edge of the PC display screen in this coordinate system. The coordinate values ​​of each edge of the PC display screen in this coordinate system can be determined based on the PC display screen resolution and the initial position of the mouse pointer. For example, ... Figure 4As shown, in this coordinate system, the right edge of the PC display screen has a coordinate of x1 on the X-axis, the left edge has a coordinate of -x2 on the X-axis, the top edge has a coordinate of y1 on the Y-axis, and the bottom edge has a coordinate of -y2 on the Y-axis. After the mouse moves, it reports its relative displacement to the PC. The PC calculates the mouse pointer's coordinates (x, y) on the PC display screen based on these coordinates. Using these coordinates (x, y), the PC can determine whether the mouse pointer has moved off the edge of the PC display screen. For example, if the mouse pointer's X-axis coordinate x is greater than x1, it indicates the mouse pointer has moved off the right edge of the PC display screen. If the mouse pointer's X-axis coordinate x is less than -x2, it indicates the mouse pointer has moved off the left edge of the PC display screen. If the mouse pointer's Y-axis coordinate y is greater than y1, it indicates the mouse pointer has moved off the top edge of the PC display screen. If the mouse pointer's Y-axis coordinate y is less than -y2, it indicates the mouse pointer has moved off the bottom edge of the PC display screen.

[0187] Once the mouse pointer moves off the edge of the PC screen (i.e., the mouse hovering condition is triggered), if the user continues to operate the PC's input device, such as the mouse, the PC can send the data of the user's input using the PC's input device to other terminals that created the virtual input device. For example, after the user inputs data using the PC's input device, the PC can intercept the received corresponding input events, such as mouse movement events, mouse press events, and mouse release events, and transmit the operation parameters contained in the intercepted input events, such as mouse operation parameters, to other terminals connected to the PC that created the virtual input device. These terminals can then respond to the PC's input device, such as the mouse. This example uses mouse events as the input event. Input events (or mouse events) may include operation parameters (such as mouse operation parameters) such as: mouse button flags (indicating which operation the user performed, such as pressing, releasing, moving, or scrolling the mouse), coordinate information (indicating the X and Y coordinates of the mouse movement when the user moves the mouse), scroll wheel information (indicating the X and Y axis distances of the scroll wheel movement when the user operates the mouse wheel), and key information (indicating which of the left, middle, or right mouse buttons the user operated).

[0188] In some embodiments, if only one device is connected to the PC, such as the mobile phone mentioned above, and the mobile phone has created a virtual input device, the PC can transmit the corresponding operation parameters to the mobile phone so that the mobile phone can respond to the PC's input device, such as a mouse. If multiple devices are connected to the PC, and some or all of these devices have created virtual input devices, the PC can display a list of options on its screen when it determines that a mouse hover condition has been triggered. This list includes the identifiers of the devices connected to the PC that have created virtual input devices (such as the mobile phone mentioned above). The PC can determine the device that will respond to the PC's input device operations based on the user's selection. If the user selects the mobile phone's identifier, the PC can send the corresponding operation parameters to the mobile phone so that the mobile phone can respond to the PC's input device operations. After receiving the corresponding operation parameters, the mobile phone can simulate the corresponding input device, such as a mouse event, and respond accordingly, i.e., the mobile phone can respond to the PC's input device operations. In this embodiment, after completing the creation of the virtual input device, the device connected to the PC can send a message indicating that the virtual input device has been successfully created to the PC. Based on the received instruction message, the PC can determine which devices connected to the PC have successfully created virtual input devices, and display the above list options accordingly.

[0189] In some other embodiments, the shuttle relationship can be pre-configured. If multiple devices are connected to the PC, and some or all of these devices have created virtual input devices, the pre-configured shuttle relationship can determine which device the mouse will shuttle to, i.e., which device will respond to the PC's input device operation. For example, if the multiple devices connected to the PC include the aforementioned mobile phone, and the mobile phone has created a virtual input device, the pre-configured shuttle relationship is that if the mouse pointer slides out from the left side (or left edge) of the PC display, the mouse will shuttle to the mobile phone. Then, when the user moves the mouse so that the mouse pointer slides across the left edge of the PC display, the PC can not only determine that the mouse shuttle has started, but also that the mouse has shuttled to the mobile phone. That is, the PC can send the corresponding operation parameters to the mobile phone so that the mobile phone can respond to the PC's input device operation. Of course, if there is only one device connected to the PC, and that device has created a virtual input device, the pre-configured shuttle relationship can also be used to determine whether the mouse will shuttle to that device. For example, the pre-configured shuttle relationship is that if the mouse pointer slides out from the left edge of the PC display, the mouse will shuttle to the mobile phone. However, if the user moves the mouse so that the mouse pointer slides across the right edge of the PC display, it can be determined that the mouse does not move to the phone. In other embodiments, the location of the device can be identified to determine which device the mouse is moving to. For example, taking the mouse as the input device, if the user presses and moves the mouse so that the mouse pointer slides across the left edge of the PC display, positioning technologies such as Bluetooth, Ultra-wideband (UWB), and ultrasound can be used to identify the location of devices around the PC. If the PC identifies the phone to its left, it can be determined that the mouse is moving to the phone.

[0190] The shuttle relationship can be pre-configured by the user through a configuration file, or a configuration interface can be provided for the user to configure the shuttle relationship in advance. For example, let's take configuring the shuttle relationship of a mobile phone through the interface. The PC receives the user's command to open the configuration interface and displays the configuration interface. This configuration interface includes PC identifiers (such as PC icons) and mobile phone identifiers (such as mobile phone icons). The user can configure the shuttle relationship by dragging these two identifiers. As an example, if the user places the mobile phone identifier to the left of the PC identifier, the PC can determine that when the mouse pointer moves across the left edge of the PC screen, the mouse will shuttle to the mobile phone. If the user places the mobile phone identifier to the right of the PC identifier, the PC can determine that when the mouse pointer moves across the right edge of the PC screen, the mouse will shuttle to the mobile phone. When there are multiple devices, the shuttle relationship of each device can be configured in advance. The following embodiments all use the determined mouse shuttle to mobile phone as an example for explanation. It should be noted that for the two implementations of determining which device the mouse will shuttle to based on the pre-configured shuttle relationship and the identified device position, the above-mentioned S301 can be executed before or after the mouse shuttle is triggered. This embodiment does not impose any specific restrictions here.

[0191] For example, when a user inputs a movement operation using a PC's input device, such as moving the mouse, the PC's keyboard and mouse module can receive the corresponding input event, such as a movement event, which can be called a mouse movement event. Since the mouse hovering condition has not yet been triggered, the hook will not intercept this input event, i.e., it will not intercept the mouse movement event, which will be transmitted to the PC's Windows system. Based on the received mouse movement event, the PC's Windows system can draw an animation of the mouse pointer movement and display it on the PC's screen. For example, as... Figure 5A As shown, as the mouse 401 moves, the PC displays an animation of the mouse pointer 403 moving on the PC's screen 402, such as... Figure 5A The movement trajectory of the mouse pointer 403 shown is as shown in trajectory 404.

[0192] As described in S301, after the keyboard and mouse sharing mode is enabled, the PC will activate input monitoring and mount a hook. While the mouse pointer moves across the PC screen, the PC's keyboard and mouse module can use input monitoring to detect the real-time coordinates of the mouse pointer on the PC screen. When the PC's keyboard and mouse module determines, based on the detected real-time coordinates of the mouse pointer on the PC screen, that the mouse pointer has passed the edge of the PC screen, it can be determined that the above-mentioned mouse movement condition has been met, indicating that the user wants to use the mouse to control other terminals. The PC's keyboard and mouse module can then determine that mouse movement has begun.

[0193] After the PC's keyboard and mouse module determines that mouse hovering has started, if the user interacts with the PC's input device, such as the mouse, the PC's keyboard and mouse module will use a hook to intercept the received input device event, such as the mouse event, and use the hook to capture the operation parameters in the intercepted input event, such as mouse operation parameters. Then, the operation parameters are transmitted to the mobile phone through the PC's transmission management module via the established connection, so that the mobile phone can make a corresponding response. For example, combined with Figure 5A The example shown uses a mouse as the input device. After the mouse pointer moves across the edge of the PC screen, the user continues to move the mouse in the same direction. The PC's keyboard and mouse module can receive movement events, such as mouse movement events. Since the mouse movement has already begun, the PC's keyboard and mouse module can use a hook to intercept (or block) it, so that the mouse movement event is not sent to the PC's Windows system, thus preventing the PC from responding to the received mouse movement event. The PC's keyboard and mouse module can also use a hook to capture the operation parameters in the mouse movement event, such as mouse operation parameters (e.g., mouse operation parameter 1, which can be the third operation parameter in this application), and send mouse operation parameter 1 to the mobile phone through the established connection via the PC's transmission management module. For example, as an example, mouse operation parameter 1 corresponding to the mouse movement event can be: a mouse button flag indicating that the user has moved the mouse, coordinate information indicating the X and Y coordinates of the mouse movement, scroll wheel information (with a value of empty), and key information (with a value of empty).

[0194] Furthermore, after the PC's keyboard and mouse module determines that mouse movement has begun, it can send movement status information (which can be the first movement status information in this application) to the mobile phone via the established connection through the PC's transmission management module. Upon receiving this information, the mobile phone can simulate a mouse pointer and display it on its screen (wherein, the mouse pointer displayed on the mobile phone screen can be the first cursor in this application). For example, after receiving this information, the mobile phone's transmission management module transmits it to the mobile phone's drag-and-drop management module and launcher. The drag-and-drop management module can then create a mouse pointer and have it displayed by the mobile phone's launcher. The PC's keyboard and mouse module can also hide the mouse pointer displayed on the PC screen, providing the user with the visual effect of the mouse pointer moving from the PC to the mobile phone.

[0195] S305. The mobile phone receives mouse operation parameter 1, simulates a mouse movement event based on mouse operation parameter 1, and displays an animation of mouse pointer movement on the mobile phone screen based on the mouse movement event.

[0196] After receiving operation parameters, such as mouse operation parameter 1, the mobile phone can simulate a movement event, such as a mouse movement event, based on mouse operation parameter 1. In response to this mouse movement event, the mobile phone can draw an animation of the mouse pointer movement and display it on the mobile phone's screen.

[0197] Generally, for applications installed on a mobile phone, the phone can display the icons of these applications on the phone's desktop (or home screen). Referring to the description in S304, the user triggers the mouse pointer to move from the PC screen to the mobile phone screen to drag the application icon displayed on the mobile phone screen from the phone to the PC. Therefore, after the user moves the mouse pointer from the PC screen to the mobile phone screen, the user will continue to move the mouse; that is, the user will use the PC mouse to input the operation of moving the mouse pointer on the mobile phone until the mouse pointer reaches the position of the application icon that the user wants to drag on the mobile phone screen.

[0198] It should be noted that the applications described in this embodiment can be embedded applications (i.e., system applications of the mobile phone, such as calculator, camera, settings, gallery, etc.) or downloadable applications (such as browser, weather, email, etc.). Embedded applications are applications provided as part of the mobile phone implementation. Downloadable applications are applications that can provide their own Internet Protocol Multimedia Subsystem (IMS) connectivity. These downloadable applications can be pre-installed on the mobile phone or third-party applications downloaded and installed by the user.

[0199] For example, after the phone's transmission management module receives operation parameters, such as mouse operation parameter 1, it can transmit this parameter to the phone's drag-and-drop management module. Because the operating systems of PCs and phones differ, the key values ​​of mouse operation parameters in their input events, such as mouse events, vary. Therefore, the phone's drag-and-drop management module can convert the received key code of mouse operation parameter 1 into a key code that the phone can recognize, based on a preset mapping relationship. Then, the phone, using a created virtual input device, can simulate an input device that the phone can recognize based on the converted key code of mouse operation parameter 1. For example, it can simulate movement events, such as mouse movement events. Based on the simulated mouse movement events, the phone's drag-and-drop management module can draw an animation of the mouse pointer movement and have the phone's launcher display the animation on the phone's screen. For example, as shown... Figure 5BAs shown, taking the calculator icon 504 displayed on the phone screen as an example, which the user wants to drag, the phone displays an animation of the mouse pointer 502 moving on the screen 501 as the mouse 401 moves. The movement trajectory of the mouse pointer 502 is shown in trajectory 503. That is, as the mouse 401 moves, the mouse pointer 502 can move along trajectory 503 to the position of the calculator icon 504.

[0200] After the mouse pointer moves to the desired draggable object, such as an application icon, the user can input a drag operation using the PC mouse. The phone can then drag the application icon based on this drag operation. This drag operation can be used to instruct the user to drag the object, such as the application icon. The drag operation can include one or more operations. For example, a drag operation can include two operations: a press operation and a move operation. Taking a mouse as the input device, the press operation can be a mouse press operation, and the move operation can be a mouse move operation. The process of a user dragging an application icon on a mobile phone using a PC mouse can include the following steps S306-S309.

[0201] S306: The PC receives a mouse press event, intercepts the mouse press event, and sends the mouse operation parameter 2 contained in the mouse press event to the mobile phone.

[0202] Among them, mouse operation parameter 2 can be the first operation parameter in this application.

[0203] S307. The mobile phone receives mouse operation parameter 2, simulates a mouse press event based on mouse operation parameter 2, and obtains application information based on the mouse press event.

[0204] Among them, a press event, such as a mouse press event, is an event received by the PC after the user presses the input device, such as the mouse, when the mouse pointer moves to the display position of the application icon. It should be noted that, taking the mouse as the input device as an example, "a mouse press operation" can be either a press of the left mouse button or a press of the right mouse button; this embodiment does not impose any specific limitations.

[0205] Combination Figure 5B As described in S305, the user wants to change the icons of applications displayed on the phone screen (e.g., ...). Figure 5BIn the example shown, the calculator icon (504) is dragged from the phone to the PC. Taking the mouse as the input device, when the user moves the mouse pointer to the location of the calculator icon (504), the user can press the mouse (e.g., the left mouse button). The PC's keyboard and mouse module can receive the press event, such as the mouse press event. Since the mouse press event is received by the PC after the mouse hovering begins, the PC's keyboard and mouse module will use a hook to intercept (or block) it, so that the mouse press event will not be sent to the PC's Windows system, thus preventing the PC from responding to the received mouse movement event. The PC's keyboard and mouse module can also use a hook to capture the operation parameters of the mouse press event, such as mouse operation parameters, referred to as mouse operation parameters 2, and send the captured mouse operation parameters 2 to the phone through the established connection via the PC's transmission management module. Mouse operation parameters 2 may include: a mouse button flag indicating that the user has pressed the mouse, coordinate information (empty value), scroll wheel information (empty value), and key information indicating that the user has operated the left mouse button.

[0206] The phone's transmission management module receives the corresponding mouse operation parameter 2 and sends it to the phone's drag-and-drop management module. Upon receiving the mouse operation parameter 2, the drag-and-drop management module converts the key code of the received parameter into a key code recognizable by the phone, based on a preset mapping relationship. It then uses a created virtual input device to simulate a mouse press event, such as a mouse button press, based on the converted key code. Afterward, the drag-and-drop management module determines whether the user intends to drag the icon of the application at the current mouse pointer position (e.g., the calculator icon). For example, after mouse-keyboard navigation begins (e.g., the phone receives navigation status information from the PC indicating the start of mouse navigation), the phone can register a listener for the mouse pointer's coordinates. Through this listener, the phone can monitor the mouse pointer's coordinates on the phone's screen in real time. In other words, the drag-and-drop management module can use the listener to determine the current coordinates of the mouse pointer. Subsequently, the phone can obtain information about the application. For example, the phone's launcher can obtain application information and send it to the phone's drag-and-drop management module. The phone's drag-and-drop management module saves the received application information. This application information may include one or more of the following: the application's name, the application's icon, the application's package name, the application's class name, the size of the application's icon, and the position of the application's icon on the phone's screen.

[0207] In addition, in this embodiment, when the phone simulates a press event, such as a mouse press event, it determines that the user has performed a press operation on the application icon. After the mouse press operation, the phone's drag management module can create an icon identical to the application icon (e.g., a drag icon, which can be the draggable object in this application) for subsequent dragging. The phone's drag management module can adjust the transparency of this drag icon appropriately, or it can leave it unchanged. Optionally, the phone's drag management module can hide the application icon displayed on the phone screen (e.g., the original icon), or it can leave it displayed on the phone screen by the phone's launcher.

[0208] S308: The PC receives a mouse movement event, intercepts the mouse movement event, and sends the mouse operation parameters 3 contained in the mouse movement event to the mobile phone.

[0209] Among them, mouse operation parameter 3 can be the second operation parameter in this application.

[0210] S309. The mobile phone receives mouse operation parameters 3, simulates mouse movement events based on mouse operation parameters 3, and displays an animation of the application icon following the mouse pointer movement on the mobile phone screen as a response to mouse press events and mouse movement events.

[0211] When a user wants to drag an application icon, as described in S307, after pressing the mouse button, the user can move the mouse without lifting their finger while continuing to press the button. At this time, the PC's keyboard and mouse module can receive movement events, such as mouse movement events. Similarly, since the PC receives this mouse movement event after mouse hovering begins, the PC's keyboard and mouse module uses a hook to intercept (or block) it, preventing the mouse movement event from being sent to the PC's Windows system, thus preventing the PC from responding to the received mouse movement event. The PC's keyboard and mouse module can also use a hook to capture the operation parameters contained in the mouse movement event, such as mouse operation parameters (e.g., mouse operation parameter 3), and send them to the mobile phone via the established connection through the PC's transmission management module.

[0212] The phone's transmission management module receives mouse operation parameter 3 and sends it to the phone's drag-and-drop management module. Upon receiving mouse operation parameter 3, the drag-and-drop management module converts it into a keycode and then uses a created virtual input device to simulate corresponding movement events, such as mouse movement events. Next, combining the simulated press and movement events (such as mouse press and mouse movement events), the drag-and-drop management module can initiate dragging of the application icon, and can also draw an animation of the application icon following the mouse pointer movement, which is then displayed on the phone's screen by the launcher.

[0213] For example, combining Figure 5B For example, after a user presses the mouse button while the mouse pointer is over the calculator icon, the user doesn't lift their finger and continues moving the mouse while the button is still pressed. Then, as... Figure 6 As shown, as the mouse 401 moves, the phone displays an animation on its screen 601 of the calculator icon 602 following the mouse pointer 603, as shown in trajectory 604. In some embodiments, the calculator icon 602 following the mouse pointer 603 can be the draggable icon described in S307, that is, an icon created (or copied) by the phone that is identical to the calculator icon displayed on the desktop. It can also be understood that when the phone does not hide the original calculator icon, after the draggable icon is dragged, the phone's screen displays not only the original calculator icon but also the draggable icon that follows the mouse pointer 603. It should be noted that in some embodiments, only the application icon may move with the mouse; that is, only the application icon is dragged. In other embodiments, the application icon and the application name are dragged together with the mouse; that is, the application icon and the application name are dragged together (e.g., ...). Figure 6 (As shown in the image). In an embodiment where the application icon and application name are dragged together, the phone, when creating the drag icon, also needs to create a name that is the same as the name of the application being dragged, for dragging purposes.

[0214] In this embodiment, during the process of dragging the icon, the mobile phone does not need to find a new place for the icon or perform desktop page turning. Instead, it executes the following steps S310-S311 so that the dragged icon can be dragged onto the PC's display screen.

[0215] S310: Mobile phone monitors the coordinates of the mouse pointer on the mobile phone screen.

[0216] S311. Based on the coordinates of the mouse pointer on the phone screen, when the phone determines that the mouse pointer has moved out of the edge of the phone screen, the phone sends a transit status information to the PC to indicate that the mouse transit has ended, and sends the application icon to the PC.

[0217] As the application icon moves across the phone's screen with the mouse pointer, the phone can determine whether the dragged application icon has been pulled off the edge of the screen. In this embodiment, the application icon being dragged off the edge of the screen can mean that a portion of the application icon is dragged off (or overflows) the screen (i.e., the proportion of the application icon overflowing the screen is greater than a preset threshold), or that the entire area of ​​the application icon is dragged off (or overflows) the screen, or that the mouse pointer slides off the edge of the screen as the application icon moves across the screen. This embodiment does not impose specific limitations. The following explanation uses the determination of whether the dragged application icon has been dragged off the screen specifically as determining whether the mouse pointer slides off the edge of the screen as an example.

[0218] As the application icon moves across the phone's screen with the mouse pointer, the phone (such as its transmission management module) can monitor the real-time coordinates of the mouse pointer on the screen (as described in S307, the real-time coordinates of the mouse pointer can be obtained using a registered listener). For example, the phone can determine the coordinates of the mouse pointer on the screen based on its initial position and relative displacement, thereby determining whether the mouse pointer has moved off the edge of the screen. The initial position of the mouse pointer can be its coordinates on the screen when the mouse begins to move, or its coordinates before the mouse begins to move. Specifically, this initial position can refer to the coordinates in a coordinate system with the top-left corner of the screen as the origin, the X-axis pointing from the top-left corner to the right edge of the screen, and the Y-axis pointing from the top-left corner to the bottom edge of the screen. The specific implementation of the phone determining whether the mouse pointer has moved off the screen edge is similar to that of a PC determining whether the mouse pointer has moved off the screen edge, and will not be detailed here.

[0219] When the phone's transmission management module determines, based on the real-time coordinates of the mouse pointer on the phone's screen, that the mouse pointer has moved across the edge of the screen, it indicates that the user wants to control other devices with the mouse. As described in S304, if the phone is connected to only one device, the PC, it indicates that the user wants to control the PC with the mouse. If the phone is connected to multiple devices, it can display a list of options, including the identifiers of all devices connected to the phone, allowing the user to select the device they want to control with the mouse. If the user selects the PC identifier, it indicates that the user wants to control the PC with the mouse. Alternatively, a routing relationship can be pre-configured in the phone to determine which device the mouse will route to, i.e., which device will respond to the mouse operation. The specific descriptions of the routing relationship configuration and application are similar to those in the above embodiments and will not be elaborated upon here.

[0220] When the phone's transmission management module determines that the user intends to control the PC using a mouse, it can determine that mouse hovering has ended. At this time, the phone's transmission management module can send hovering status information (which can be the second hovering status information described in this application) to the PC through the established connection, indicating the end of mouse hovering. The phone's transmission management module can also indicate the end of mouse hovering to the phone's drag management module, so that the drag management module can send drag data to the PC through the established connection via the phone's transmission management module. This drag data may include an application icon. Before sending the application icon to the PC, the phone can also serialize the application icon; that is, the application icon sent to the PC by the phone can be data obtained after serialization. If the application name and application icon are dragged together, the phone also needs to send the application name to the PC.

[0221] Additionally, once the mouse pointer moves across the edge of the phone screen, indicating the mouse hover is complete, the dragging of the application icon on the screen ends. The phone's drag management module can also indicate the end of the mouse hover to the phone's launcher, so that the launcher can display a drag-end animation on the phone's screen, such as showing the dragged icon moving along a trajectory from the edge of the phone screen (e.g.,...). Figure 6 The animation shown (track 604, direction opposite to track 604) returns to the original position. The phone does not rearrange the app icons displayed on the screen.

[0222] It should be noted that the above embodiment illustrates the scenario where the phone sends drag data, such as the application icon, to the PC after the mouse hovering is completed. In other embodiments, the phone may temporarily cache drag data, such as object information (including the application icon), at the start of dragging. After the mouse hovering is completed, the phone sends hovering status information to the PC indicating the end of the mouse hovering. Upon receiving this hovering status information, the PC may send a message to the phone requesting drag data, and the phone, upon receiving this message, then sends the drag data to the PC.

[0223] S312, The PC receives a mouse movement event and, based on the mouse movement event and the received application icon, displays an animation on the PC's screen showing the application icon moving with the mouse pointer.

[0224] After the PC's transmission management module receives the mouse hover status information from the mobile phone, indicating the end of mouse hovering, it can indicate the end of mouse hovering to the PC's drag management module and the PC's keyboard and mouse module. The PC can then determine that the user needs to switch back to PC-based mouse operation response. Afterward, the PC can uninstall the hook, i.e., cancel the interception of input events, such as mouse events. Subsequently, when the user operates the PC's input devices, the PC's keyboard and mouse module will not intercept received input events, but will instead send the received input events to the PC's Windows system so that the PC's Windows system can respond to the input events. The PC's keyboard and mouse module can also redisplay the mouse pointer on the PC screen.

[0225] For example, consider a mouse as the input device. The user continues to press and move the mouse. The PC receives movement events, such as mouse movement events. Since the hook has been uninstalled, the PC's keyboard and mouse module sends the received mouse movement event to the PC's Windows system for a corresponding response. For instance, the PC can draw the application's icon based on the mouse movement event and the received application icon, and display the application's icon in a window. The PC will also animate the application's icon as it moves with the mouse pointer and display this animation on the PC's screen. If the application's name and icon are dragged together, the PC also needs to draw the application's name, display it in a window, and move it with the mouse pointer. For example, combining... Figure 6 ,like Figure 7 As shown, as the mouse 401 moves, the PC displays an animation on the PC screen 701 of the calculator icon 702 following the mouse pointer 703, as shown in trajectory 704. That is to say, combined with... Figure 6 and Figure 7As shown, by long-pressing and moving the mouse, users can visually drag an application icon (such as a calculator icon) from the phone screen to the PC screen. Of course, if the phone serializes the application icon before sending it to the PC, the PC can obtain the icon by deserializing the received data.

[0226] Additionally, in S311, the mobile phone can send the size of the application icon and its display position on the mobile phone screen obtained in S307 to the PC. Based on this data, the PC can determine the size and initial position of the window used to display the application icon using preset rules. As an example, the preset rules could be that the window size is the same as the size of the application icon obtained in S307, and the initial position of the window is the same as the display position of the application icon obtained in S307 on the mobile phone screen (e.g., if the application icon obtained in S307 is displayed in the lower right corner of the mobile phone screen, then the initial position of the window is the lower right corner of the PC screen). Alternatively, the window size can be twice the size of the application icon obtained in S307, and the initial position of the window is the same as the display position of the application icon obtained in S307 on the mobile phone screen, etc. This embodiment of the application does not specifically limit the rules herein.

[0227] S312 is an optional step.

[0228] S313: The PC receives a mouse lift-off event and displays the application interface on the PC screen based on the mouse lift-off event.

[0229] After a user drags the icon of an application they want to drag (such as a calculator icon) onto the PC's screen, the user can input a drag-release operation to end the dragging of the application icon. For example, the user can lift their finger (i.e., the drag-release operation can be a finger lift-up operation, or stopping a long-press operation on the mouse). The user can also stop moving the mouse. The PC's keyboard and mouse module can receive this lift-up event, such as a mouse lift-up event. Similarly, since the hook has been uninstalled at this time, the PC's keyboard and mouse module will send the received lift-up event, such as a mouse lift-up event, to the PC's Windows system for response. For example, in response to the mouse lift-up event, the PC can display the interface of the dragged application at the location where the mouse pointer stopped moving on the PC's screen. For example, combined with Figure 7 Examples, such as Figure 8As shown, when a user drags the calculator icon onto the PC screen, the user lifts their finger and stops moving the mouse. In response, the PC can display the calculator interface 802 on the PC screen 801. Alternatively, for a mobile phone, the interface displayed on the phone may remain unchanged after the user lifts their finger and stops moving the mouse, such as... Figure 8 As shown in (a), desktop 803 is still displayed, and the phone can also display the calculator interface, as shown in [example image]. Figure 8 As shown in (b) above, the phone displays the calculator interface 804.

[0230] As an example, the specific implementation of displaying the application's interface on the PC's screen based on a mouse lift-off event can be as follows: After the PC's keyboard and mouse module receives a lift-off event, such as a mouse lift-off event, the PC's keyboard and mouse module can send a message to the mobile phone through the PC's transmission management module via a connection established with the mobile phone. This message can be a request message to request the application's interface, or it can be an indication message to indicate the end of dragging the application icon. After receiving the message, the mobile phone can open the application based on the application's package name and class name obtained in S307, and send the opened application's interface to the PC through the established connection, so that the PC can display the application's interface on its screen. For example, the PC and mobile phone can use wireless projection (Miracast) to display the application's interface on the PC's screen. That is, after receiving the above message, the mobile phone can integrate all layers of the obtained application's interface into a video stream (or screen recording data), encode it, and send it to the PC via the Real-Time Streaming Protocol (RTSP). After receiving the video stream, the PC can decode and play it to display the application's interface on the PC's screen. For example, a mobile phone can use Distributed Multimedia Protocol (DMP) technology to display the application's interface on a PC screen. After receiving the aforementioned message, the mobile phone can create a virtual display. Then, the mobile phone can move the obtained application interface rendering to this virtual display. The mobile phone can then record the application interface rendered on the virtual display. In this way, the mobile phone can obtain the screen recording data of the application's interface. The mobile phone can encode the obtained screen recording data and send it to the PC. The PC can receive the corresponding data, decode it, and obtain the screen recording data. Based on the decoded screen recording data, the PC can then display the application's interface on the PC screen.

[0231] The method provided in this embodiment allows users to control all terminals using the same input device, such as a mouse, in scenarios where multiple terminals are used collaboratively. Users can drag and drop application icons displayed on one terminal to other terminals by moving the mouse, increasing interaction between terminals and improving the user experience.

[0232] Figure 9 This is a flowchart illustrating another cross-device object drag-and-drop method provided in this application embodiment. In this embodiment, the process of a user using a shared mouse on a PC to drag the interface of an application displayed on a mobile phone (the application interface being the dragged object) from the mobile phone to the PC is described. Figure 9 As shown, the method may include the following S901-S913.

[0233] S901: The mobile phone establishes a connection with the PC.

[0234] S902: The PC receives mouse movement events and displays an animation of mouse pointer movement on the PC's screen based on the mouse movement events.

[0235] S903, PC monitors the coordinates of the mouse pointer on the PC screen.

[0236] S904: Based on the coordinates of the mouse pointer on the PC screen, when the mouse pointer moves out of the edge of the PC screen, the PC intercepts the mouse movement event and sends the mouse operation parameter 1 contained in the mouse movement event to the mobile phone.

[0237] It should be noted that for descriptions of other related content in S901-S904, please refer to... Figure 3 The descriptions of the corresponding contents in S301-S304 of the illustrated embodiment are not repeated here.

[0238] S905: The mobile phone receives mouse operation parameter 1, simulates a mouse movement event based on mouse operation parameter 1, and displays an animation of mouse pointer movement on the mobile phone screen based on the mouse movement event.

[0239] Among them, mouse operation parameter 1 is the third operation parameter in this application.

[0240] After receiving mouse operation parameter 1, the mobile phone can simulate a movement event, such as a mouse movement event, based on this parameter. In response to this mouse movement event, the mobile phone can draw an animation of the mouse pointer movement and display it on the phone's screen. In other words, by moving the mouse, the user can trigger the mouse pointer to move from the PC screen to the mobile phone screen.

[0241] Taking the example where a user moves the mouse pointer from the PC screen to the phone screen to drag the currently displayed application interface from the phone to the PC. In this embodiment, the operation that triggers the application interface windowing can be a predefined operation, such as a long press on the status bar. Therefore, after the user moves the mouse pointer from the PC screen to the phone screen, the user will continue to move the mouse until the mouse pointer reaches the position of the status bar displayed on the phone screen.

[0242] For example, consider a user wanting to drag the calculator interface from their phone to their PC screen. Figure 10 As shown, the phone's screen currently displays a calculator interface 1001. After the user moves the mouse 1002, triggering the mouse pointer to move from the PC screen to the phone's screen, the user will continue to move the mouse 1002. As the mouse 1002 moves, the phone displays a corresponding animation of the mouse pointer 1003 moving on its screen, as shown in trajectory 1004. That is, as the mouse 1002 moves, the mouse pointer 1003 can move along trajectory 1004 to the position of the status bar 1005 displayed on the phone.

[0243] S906: The PC receives mouse press events and mouse release events, intercepts these events, and sends the mouse operation parameters 2 for the mouse press event and 3 for the mouse release event to the mobile phone.

[0244] Wherein, mouse operation parameter 2 is the fourth operation parameter in this application, and mouse operation parameter 3 is the fifth operation parameter in this application.

[0245] S907: The mobile phone receives mouse operation parameters 2 and 3, simulates a mouse press event based on mouse operation parameter 2, simulates a mouse release event based on mouse operation parameter 3, displays a window on the mobile phone screen based on the mouse press and mouse release events, the window includes a screenshot of the application interface, and obtains the window information.

[0246] Combination Figure 10 And according to the description of S905, the user wants to display the application interface (e.g., ...) on the phone screen. Figure 10In the example shown, the calculator interface 1001 is dragged from the phone to the PC, and the operation that triggers the application interface windowing is a long press on the status bar. When the user moves the mouse pointer to the status bar, the user can press the mouse (e.g., the left mouse button) and release the finger after a certain period of time. The PC's keyboard and mouse module can receive and intercept the press event (e.g., mouse press event) and release event (e.g., mouse release event). The PC's keyboard and mouse module can also capture the operation parameters (e.g., mouse operation parameter 2 and mouse operation parameter 3) of the intercepted mouse press and release events through the PC's transmission management module and send them to the phone through the established connection.

[0247] The phone's transmission management module receives mouse operation parameters 2 and 3 and sends them to the phone's drag-and-drop management module. Upon receiving these parameters, the drag-and-drop management module converts the key codes of mouse operation parameter 2 and 3 into key codes recognizable by the phone, based on a preset mapping relationship. Then, using a created virtual input device, it simulates a mouse press event (e.g., a mouse down event) and a mouse release event (e.g., a mouse up event) based on the converted key codes of mouse operation parameter 3. Afterward, the drag-and-drop management module determines the user's intention to drag the currently displayed application interface (e.g., the calculator interface) based on the current mouse pointer position. In response to these mouse press and release events, the phone's system UI sends a status bar event to the phone's launcher. After receiving the status bar event, the phone's launcher can window the currently displayed application's interface, that is, display a window on the phone's screen. This window includes the application's interface, such as a screenshot of the application interface (e.g., a screenshot of a calculator interface). The window size can be a preset value. Additionally, the phone (e.g., the phone's launcher) can also apply background blurring. In this embodiment, the background can be the application's interface or the phone's desktop; this embodiment is not limited to these. For example, combined with... Figure 10 ,like Figure 11 As shown, after simulating mouse press and release events, the phone displays window 1101 on its screen. This window 1101 includes a screenshot of the calculator interface 1001. The phone also applies background blurring. Figure 11 The example shown is an interface 1001 with a computer background.

[0248] The mobile phone (such as the phone's launcher) can also obtain window information (which is the information of the object in this application). The window information may include one or more of the following: the name of the application in the window, the application's package name, the application's class name, a screenshot of the application's interface, the window's size, and the window's position on the mobile phone's display screen.

[0249] S908 and PC receive mouse movement events and mouse press events, intercept the mouse movement events and mouse press events, and send the mouse operation parameters 4 of the mouse movement event and the mouse operation parameters 5 of the mouse press event to the mobile phone.

[0250] S909: The mobile phone receives mouse operation parameters 4 and 5, simulates mouse movement events based on mouse operation parameter 4, simulates mouse press events based on mouse operation parameter 5, and displays an animation on the mobile phone screen showing the window moving with the mouse pointer based on the mouse movement and mouse press events.

[0251] Mouse operation parameter 4 can be the first operation parameter in this application. Mouse operation parameter 5 can be the second operation parameter in this application.

[0252] When a user wants to drag the application interface, they can move the mouse to move the mouse pointer from the phone's status bar to the S907's window. Then, the user can long-press the mouse button while simultaneously moving it. At this time, the PC's keyboard and mouse module receives both press events (e.g., mouse press events) and movement events (e.g., mouse move events). Similarly, the PC's keyboard and mouse module intercepts these events, captures the corresponding operation parameters (e.g., mouse operation parameter 4 and mouse operation parameter 5), and sends them to the phone via the established connection through the PC's transmission management module.

[0253] The phone's transmission management module receives mouse operation parameters 4 and 5 and sends them to the phone's drag-and-drop management module. Upon receiving these parameters, the drag-and-drop management module converts the key codes of mouse operation parameter 4 and 5 into key codes recognizable by the phone, based on a preset mapping relationship. Then, using a created virtual input device, it simulates a press event (e.g., a mouse press event) based on the converted key codes of mouse operation parameter 4, and a movement event (e.g., a mouse move event) based on the converted key codes of mouse operation parameter 5. In response, the drag-and-drop management module can initiate a drag-and-drop operation on a window and draw an animation of the window following the mouse pointer, which is then displayed on the phone's screen by the launcher. For example, combined with... Figure 11Users can move the mouse to trigger the mouse pointer to move from the status bar on the phone screen. Figure 11 The position of the middle window. Then, as... Figure 12 As shown, the user can long-press the mouse 1201 and move the mouse 1201 simultaneously. As the mouse 1201 moves, the phone displays an animation on the phone screen of window 1202 (which includes a screenshot of the calculator interface) following the movement of the mouse pointer 1203, as shown in trajectory 1204.

[0254] As the window moves with the mouse pointer, the phone can determine whether the dragged window has been dragged off the edge of the phone screen. In this embodiment, the window being dragged off the edge of the phone screen can mean that a portion of the window is dragged off (or overflows) the phone screen (i.e., the proportion of the window overflowing the screen is greater than a preset threshold), or the entire area of ​​the window is dragged off (or overflows) the phone screen, or the mouse pointer slides off the edge of the phone screen as the window moves with the mouse pointer. This embodiment does not impose specific limitations here. The following explanation uses the determination of whether the dragged window has been dragged off the edge of the phone screen specifically as determining whether the mouse pointer slides off the edge of the phone screen as an example.

[0255] S910: Mobile phone monitors the coordinates of the mouse pointer on the mobile phone screen.

[0256] S911: Based on the coordinates of the mouse pointer on the phone screen, when the mouse pointer moves out of the edge of the phone screen, the phone sends a transit status information to the PC to indicate the end of the mouse transit, and sends a screenshot of the application interface to the PC.

[0257] Before sending the application's screenshot to the PC, the mobile phone can also perform serialization processing on it. In other words, the application's screenshot sent from the mobile phone to the PC can be data obtained after serialization processing. The shuttle state information in S911 can be the second shuttle state information in this application.

[0258] S912, the PC receives a mouse movement event, and based on the mouse movement event and the received screenshot of the application interface, displays an animation on the PC's screen showing the screenshot of the application interface following the movement of the mouse pointer.

[0259] For example, the user continues to press and hold the mouse while moving it. The PC can receive movement events, such as mouse movement events. Based on this mouse movement event and a received screenshot of the application's interface, the PC can draw a screenshot of the application's interface and display it in a window. The PC will also animate the application's interface screenshot as the mouse pointer moves and display this animation on the PC's screen. For example, combined with... Figure 12 ,like Figure 13 As shown, as the mouse 1301 moves, the PC displays an animation on its screen showing the application's interface screenshot 1302 following the mouse pointer 1303, as shown in trajectory 1304. Additionally, after the mouse pointer passes the edge of the phone's screen, the dragging of the application's interface on the phone's screen ends, and the phone can display a drag-end animation on its screen. For example, the phone might display an animation of restoring the application's interface (or a desktop restoration animation). Figure 13 As shown, after the drag-and-drop operation on the phone is completed, the phone displays the calculator interface 1305.

[0260] In other words, combining Figure 12 and Figure 13 As shown, by long-pressing and moving the mouse, users can visually drag the interface of an application (such as a calculator) displayed on their phone screen to the PC screen. Of course, if the phone serializes the application's screenshot before sending it to the PC, the PC can obtain the screenshot by deserializing the received data.

[0261] Additionally, in S911, the mobile phone can send the window size and its display position on the mobile phone screen obtained in S907 to the PC. Based on this data, the PC can determine the size and initial position of the window used to display the application's interface screenshot using preset rules. As an example, the preset rules could be that the window size is the same as the window size obtained in S907, and the initial position of the window is the same as the window's display position on the mobile phone screen obtained in S907. Alternatively, the window size could be twice the size of the window obtained in S907, and the initial position of the window could be the same as the window's display position on the mobile phone screen obtained in S907, etc. This application embodiment does not specifically limit the rules herein.

[0262] S913: The PC receives a mouse lift-off event and displays the application's interface on the PC's screen based on the mouse lift-off event.

[0263] When a user drags the interface of an application (such as a calculator) to the PC screen, the user can lift their finger (i.e., stop holding down the mouse button) and stop moving the mouse. The PC receives this lift-off event, such as a mouse lift-off event. In response to this mouse lift-off event, the PC can display the application's interface on the PC screen at the location where the mouse pointer stopped moving. For example, combined with... Figure 13 Examples, such as Figure 14 As shown, when a user drags a screenshot of the calculator interface onto the PC screen, the user lifts their finger and stops moving the mouse. In response, the PC can display the calculator interface 1401 on the PC screen.

[0264] As an example, the specific implementation of displaying an application's interface on the PC screen based on a mouse lift-off event could be as follows: After the PC's keyboard and mouse module receives a mouse lift-off event, it can send a request message to the mobile phone via the PC's transmission management module through a connection established with the mobile phone. This request message requests the application's interface. Upon receiving this request message, the mobile phone can retrieve the application's interface based on the application name, package name, and class name obtained from the window information in S907, and send it to the PC through the established connection, so that the PC can display the application's interface on its screen. For example, the PC and mobile phone can use Miracast to display the application's interface on the PC screen. Alternatively, the mobile phone can use DMP technology to display the application's interface on the PC screen.

[0265] It should be noted that the specific descriptions of other contents in each step of this embodiment are different from those in the previous one. Figure 3 The descriptions of the corresponding contents in the illustrated embodiments are similar, and will not be repeated here.

[0266] The method provided in this embodiment allows users to control all terminals using the same input device, such as a mouse, in scenarios where multiple terminals are used collaboratively. Users can drag and drop the interface of an application displayed on one terminal to other terminals by moving the mouse, which increases the interaction between terminals when multiple terminals are used collaboratively and improves the user experience.

[0267] Understandably, current multi-screen collaboration for multiple terminals, such as PC and mobile phone collaborative work, requires dragging and dropping content based on projecting the mobile phone's interface onto the PC's screen. Furthermore, the dragging and processing of content are all handled by the PC's hardware. Mobile phones are typically screen-off, and their hardware capabilities (such as displays and styluses) cannot participate in collaborative work. This makes it difficult for users to fully utilize the characteristics of different terminals involved in collaborative work, maximizing their strengths and minimizing their weaknesses to maximize work efficiency. Moreover, the mobile phone's screen projected onto the PC also takes up display space on the PC screen, impacting the efficiency of multi-terminal collaborative use. The cross-device object dragging method and device provided in this embodiment, without initiating screen projection, utilizes keyboard and mouse sharing technology, allowing users to use input devices such as touchpads and mice to drag and drop text or files (or objects) between multiple terminals involved in collaborative work. Users can also use these terminals to process the transferred content.

[0268] In some other embodiments, the following is combined Figure 1 and Figure 2B Taking the first terminal 101 as a PC, the second terminal 102 as a tablet computer, and the input device 102-1 as a mouse as an example, the cross-device object dragging method provided in this application embodiment will be described in detail with reference to the accompanying drawings.

[0269] Figure 15 This is a flowchart illustrating a cross-device object drag-and-drop method provided in this application embodiment. Specifically, this embodiment uses a user's mouse on a PC to drag and drop content (the dragged object) to a tablet computer, providing a detailed explanation of the method. Figure 15 As shown, the method may include the following S1501-S1510.

[0270] S1501, The tablet computer establishes a connection with the PC.

[0271] Once the tablet and PC successfully establish a connection, they can be used together. To improve the efficiency of this collaborative use, users can use a single input device, such as a PC mouse, to control both the PC and the tablet.

[0272] Furthermore, since the mouse is an input device for the PC, after other terminals establish a connection with the PC, the user can generally temporarily control the PC using the mouse. In this embodiment, after the PC enables keyboard and mouse sharing mode, it can also trigger other terminals that have established a connection with the PC and created virtual input devices to respond to mouse operations when the mouse travel condition is met. For example, the mouse travel condition can be that the mouse pointer displayed on the PC screen moves across the edge of the PC screen.

[0273] It should be noted that the specific description of S1501 in this embodiment can be found in the specific description of the corresponding content in another embodiment S301 of this application, and will not be repeated here.

[0274] For example, if a user wants to transfer content from a PC to a tablet computer by dragging and dropping, and then continue dragging and dropping on the tablet computer, the process of triggering other terminals connected to the PC that have created virtual input devices, such as a tablet computer, to respond to mouse operations may include the following steps S1502-S1506.

[0275] S1502, The PC receives mouse operations on the selected content.

[0276] The above content can be text (or text), a file, or a folder. Files can include one or more of the following formats: Word documents, Excel workbooks, PowerPoint presentations, bitmaps, image files, plain text files, sound files, video files, Flash animation files, web page files, compressed files, etc.

[0277] Furthermore, in this embodiment, the selected content can be one or multiple items. For example, selecting two Word documents. Another example is selecting one Word document and one image file. Yet another example is selecting two folders.

[0278] S1503: The PC receives mouse press events and mouse move events, initiates drag events based on the mouse press events and mouse move events, and displays an animation of the shadow of the content moving with the mouse pointer on the PC's screen.

[0279] The PC's mouse pointer can be the second cursor in this application.

[0280] S1504, PC monitors the coordinates of the mouse pointer on the PC screen.

[0281] S1505: Based on the coordinates of the mouse pointer on the PC screen, when the mouse pointer is determined to have moved beyond the edge of the PC screen, the PC intercepts the mouse movement event and sends the mouse operation parameters contained in the mouse movement event to the tablet computer.

[0282] S1506: The PC obtains a bitmap of the drag event content and shadow, and sends the bitmap of the drag event content and shadow to the tablet computer.

[0283] The drag-and-drop event content is used to drag and drop connected devices, such as tablet computers, initiating the drag-and-drop event. For example, when the content in S1502 is text, the drag-and-drop event content can include that text. As another example, when the content in S1502 is a file or folder, the drag-and-drop event content can include the file path (e.g., a Uniform Resource Identifier, URI). The drag-and-drop data in this application can include the drag-and-drop event content and a bitmap of the shadow, which can be used to drag and drop connected devices, such as tablet computers, to display an animation of the object moving with the mouse pointer on their screens.

[0284] In this embodiment, after the keyboard and mouse sharing mode is enabled, when a user wants to transfer content from the PC to another terminal connected to the PC, such as a tablet computer, by dragging and dropping, the user can use the PC's input device to select the content to be dragged. Subsequently, when the PC's mouse pointer is displayed on the content, the user can input a drag operation, so that the PC can drag the corresponding object, i.e., the content (such as the selected content in S1502), according to the drag operation.

[0285] The drag-and-drop operation can be an action used to instruct the initiation of a drag-and-drop event on the selected content. This drag-and-drop operation can include one or more actions. For example, a drag-and-drop operation may include two actions: a press action and a move action. Taking a mouse as the input device, the press action can be a mouse press operation, and the move action can be a mouse move operation. The user can press and move the mouse (i.e., use the PC's mouse to input mouse press and mouse move actions) to trigger the PC's Windows system to initiate a drag-and-drop event on the content, so that the content (such as its shadow) can move on the PC's screen as the PC's mouse pointer moves. The shadow can also be called a drag shadow.

[0286] As content (such as its shadow) moves with the mouse pointer, the PC can determine whether the dragged content (such as its shadow) has been dragged off the edge of the PC display screen. When the content (such as its shadow) is dragged off the edge of the PC display screen, a mouse hovering condition can be triggered. In this embodiment, the content being dragged off the edge of the PC display screen can mean that a portion of the content (such as its shadow) is dragged off (or overflows) the PC display screen (i.e., the proportion of the window overflowing the display screen is greater than a preset threshold), or it can mean that the entire area of ​​the content (such as its shadow) is dragged off (or overflows) the PC display screen, or it can mean that the mouse pointer slides off the edge of the PC display screen as the content (such as its shadow) moves with the mouse pointer. This embodiment does not impose specific limitations here. The following explanation uses the determination of whether the dragged content (such as its shadow) has been dragged off the edge of the PC display screen specifically as the determination of whether the mouse pointer slides off the edge of the PC display screen as an example.

[0287] As an example, a user can trigger the mouse hovering condition by continuously moving the mouse in the same direction, causing the mouse pointer displayed on the PC screen to slide across (or out of) the edge of the PC screen.

[0288] For example, the PC can determine the coordinate position of the mouse pointer on the PC display screen based on the initial position and relative displacement of the mouse pointer, thereby determining whether the mouse pointer has slipped off the edge of the PC display screen.

[0289] The initial position of the mouse pointer can be the coordinates of the mouse pointer on the PC screen when the mouse begins to move, or the coordinates of the mouse pointer on the PC screen before the mouse begins to move. Specifically, this initial position can refer to the coordinates in a coordinate system with the top-left corner of the PC screen as the origin, the X-axis pointing from the top-left corner to the right edge of the PC screen, and the Y-axis pointing from the top-left corner to the bottom edge of the PC screen. For example, the specific process by which the PC determines whether the mouse pointer has slipped off the edge of the PC screen can be: combining... Figure 4 PC can establish a coordinate system with this initial coordinate position as the origin (e.g., Figure 4 The coordinate system shown (position o) has an X-axis pointing from the origin o to the right edge of the PC display screen, and a Y-axis pointing from the origin o to the top edge of the PC display screen. The PC can determine the coordinate values ​​of each edge of the PC display screen in this coordinate system. The coordinate values ​​of each edge of the PC display screen in this coordinate system can be determined based on the PC display screen resolution and the initial position of the mouse pointer. For example, ... Figure 4As shown, in this coordinate system, the right edge of the PC display screen has a coordinate of x1 on the X-axis, the left edge has a coordinate of -x2 on the X-axis, the top edge has a coordinate of y1 on the Y-axis, and the bottom edge has a coordinate of -y2 on the Y-axis. After the mouse moves, it reports its relative displacement to the PC. The PC calculates the mouse pointer's coordinates (x, y) on the PC display screen based on these coordinates. Using these coordinates (x, y), the PC can determine whether the mouse pointer has moved off the edge of the PC display screen. For example, if the mouse pointer's X-axis coordinate x is greater than x1, it indicates the mouse pointer has moved off the right edge of the PC display screen. If the mouse pointer's X-axis coordinate x is less than -x2, it indicates the mouse pointer has moved off the left edge of the PC display screen. If the mouse pointer's Y-axis coordinate y is greater than y1, it indicates the mouse pointer has moved off the top edge of the PC display screen. If the mouse pointer's Y-axis coordinate y is less than -y2, it indicates the mouse pointer has moved off the bottom edge of the PC display screen.

[0290] Once the mouse pointer moves off the edge of the PC screen—that is, the mouse hover condition is triggered—the user can use the PC's input device to control other terminals connected to the PC that have created virtual input devices. In other words, after the mouse hover condition is triggered, the PC can send the data of the user's input using the PC's input device to other terminals that have created virtual input devices. For example, if the user continues to move the mouse in the same direction, the PC can intercept the received movement events, such as mouse movement events, and transmit the operation parameters contained in the mouse movement events, such as mouse operation parameters, to other terminals connected to the PC that have created virtual input devices, so that those terminals can continue the drag event.

[0291] It should be noted that in this embodiment, if there is only one device connected to the PC, such as the tablet computer mentioned above, and the tablet computer creates a virtual input device, the PC can transmit the corresponding operation parameters to the tablet computer so that the tablet computer can continue the drag-and-drop event.

[0292] If multiple devices are connected to the PC, and some or all of these devices have established virtual input devices, in some embodiments, the PC can display a list of options on its screen when the mouse shuttle condition is triggered. This list includes identifiers of the devices connected to the PC that have created virtual input devices (such as the tablet computer identifier mentioned above). The PC can determine the device responsible for continuing the drag-and-drop event based on the user's selection. For example, if the user selects the tablet computer identifier, the PC can send the corresponding operation parameters to the tablet computer so that the tablet computer can continue the drag-and-drop event. After creating the virtual input device, the device connected to the PC can send a successful virtual input device creation indication message to the PC. The PC can then determine which devices connected to the PC have successfully created virtual input devices based on the received indication message and display the list of options accordingly. In other embodiments, shuttle relationships can be pre-configured. If multiple devices are connected to the PC, and some or all of these devices have established virtual input devices, the device responsible for continuing the drag-and-drop event can be determined based on the pre-configured shuttle relationships. For example, if a tablet computer is among the multiple devices connected to a PC, and the tablet computer creates a virtual input device, and the pre-configured navigation relationship is that the mouse pointer slides out from the left side (or left edge) of the PC display, then the device implementing the drag-and-drop connection is determined to be the tablet computer. Therefore, when the user presses and moves the mouse, causing the mouse pointer to slide across the left edge of the PC display, the PC can not only determine that mouse navigation has begun, but also that the device implementing the drag-and-drop connection is the tablet computer. Of course, if only one device is connected to the PC, and that device creates a virtual input device, the pre-configured navigation relationship can also be used to determine whether the device implementing the drag-and-drop connection is that device. For example, if the pre-configured navigation relationship is that the mouse pointer slides out from the left edge of the PC display, then the mouse navigates to the tablet computer. However, if the user presses and moves the mouse, causing the mouse pointer to slide across the right edge of the PC display, then the mouse cannot navigate to the tablet computer. In some other embodiments, the device implementing the drag-and-drop connection can be determined by identifying the device's location. For example, if a user presses and moves the mouse so that the mouse pointer slides across the left edge of the PC screen, positioning technologies such as Bluetooth, Ultra-wideband (UWB), and ultrasound can be used to identify the location of devices around the PC. If the PC recognizes that there is a tablet computer to its left, it can determine that the device that performs the drag-and-drop operation is the tablet computer.

[0293] The shuttle relationship can be pre-configured by the user through a configuration file, or a configuration interface can be provided for the user to configure the shuttle relationship in advance. For example, let's take the example of configuring the shuttle relationship between a tablet and a computer through an interface. The PC receives the user's command to open the configuration interface and displays the interface. This interface includes the PC's identifier (e.g., a PC icon) and the tablet's identifier (e.g., a tablet icon). The user can configure the shuttle relationship by dragging these two identifiers. As an example, if the user places the tablet's identifier to the left of the PC's identifier, the PC can determine that when the mouse pointer moves across the left edge of the PC's display, the device that enables the drag-and-drop continuation is the tablet. If the user places the tablet's identifier to the right of the PC's identifier, the PC can determine that when the mouse pointer moves across the right edge of the PC's display, the device that enables the drag-and-drop continuation is the tablet. When multiple devices exist, the shuttle relationship for each device can be pre-configured. The following embodiments all use the tablet as the device that enables the drag-and-drop continuation. It should be noted that for the two implementations of determining the device for drag-and-drop continuation based on the pre-configured shuttle relationship and the identified device position, the above-mentioned S1501 can be executed before the mouse shuttle is triggered or after the mouse shuttle is triggered. This embodiment does not impose specific restrictions here.

[0294] For example, in combination Figure 2B After a user selects content to drag using the mouse, the PC's keyboard and mouse module receives the corresponding operation, such as the mouse action of selecting the content. The user can move the mouse, causing the PC's mouse pointer to appear over the selected content. Then, if the user presses and moves the mouse (i.e., uses the PC's mouse input for both pressing and moving actions), the PC's keyboard and mouse module receives the corresponding press and move events. At this point, since the mouse hover condition has not yet been triggered, the attached hook does not intercept input events; the mouse press and move events are transmitted to the PC's Windows system. Based on the received mouse press and move events, the PC's Windows system can initiate a drag event for that content and draw the content, such as an animation of the content's shadow following the mouse pointer movement (or a drag animation), displayed on the PC's screen. For example, as shown... Figure 16 As shown, the user wants to drag and drop image 1601 from the PC to the tablet, and then continue dragging and dropping on the tablet. The user selects image 1601 using mouse 1602, then presses and moves mouse 1602. As mouse 1602 moves, the PC displays an animation on its screen 1603 showing the shadow 1606 of image 1601 moving with the mouse pointer 1604, as shown. Figure 16The shadow 1606 of image 1601 in the diagram shows the dragging trajectory as the mouse pointer 1604 moves, as shown in trajectory 1605. It should be noted that in this embodiment, the operation of selecting content, such as the mouse operation for selecting content in S1502, is optional. For example, if the user wants to drag a file or a folder, they can skip the mouse operation for selecting the content and instead, perform a press and move operation when the mouse pointer is displayed on the file or folder to initiate a drag event for that file or folder.

[0295] As described in S1501, after the keyboard and mouse sharing mode is enabled, the PC will activate input monitoring and mount a hook. After a drag event is initiated, the mouse pointer will move on the PC screen. The PC's keyboard and mouse module can use input monitoring to detect the real-time coordinate position of the mouse pointer on the PC screen. When the PC's keyboard and mouse module determines that the mouse pointer has slipped off the edge of the PC screen based on the detected real-time coordinate position of the mouse pointer, it can determine that the above-mentioned mouse movement condition is met. At this time, the PC's keyboard and mouse module can determine that mouse movement has begun.

[0296] After the PC's keyboard and mouse module determines that mouse movement has begun, it can send movement status information (which can be the first movement status information in this application) to the tablet computer via the PC's transmission management module and the established connection. Upon receiving this information, the tablet computer can simulate a mouse pointer and display it on its screen (the mouse pointer displayed on the tablet computer can be the first cursor in this application). The PC's keyboard and mouse module can also hide the mouse pointer displayed on the PC screen. Of course, objects that move with the mouse pointer, such as the object's shadow, will also be hidden. For example, combined with... Figure 16 As the mouse 1602 moves, after the mouse pointer 1604 slides across the edge of the PC display 1603, the PC hides the shadow 1606 of the image 1601 displayed on the PC display 1603 and the mouse pointer 1604. Additionally, the tablet displays the mouse pointer on its screen, giving the user the visual effect of the mouse pointer moving from the PC to the tablet.

[0297] After the PC's keyboard and mouse module determines that mouse navigation has begun, if the user interacts with the mouse, the module can use a hook to intercept the corresponding input event, such as a mouse event, and capture the operation parameters within the intercepted event, such as mouse operation parameters. These mouse operation parameters can include: mouse button flags (indicating whether the user pressed, released, moved, or scrolled the mouse), coordinate information (indicating the X and Y coordinates of mouse movement when the user moved the mouse), scroll wheel information (indicating the X and Y axis distances of scrolling when the user scrolled the mouse wheel), and key information (indicating which of the left, middle, or right mouse buttons the user activated). The PC's keyboard and mouse module can also transmit the captured operation parameters, such as mouse operation parameters, to the tablet computer via an established connection through the PC's transmission management module, allowing the tablet computer to respond accordingly. For example, continuing to combine... Figure 16 In the example shown, after the mouse pointer moves across the edge of the PC screen, the user continues to move the mouse in the same direction. The PC's keyboard and mouse module can receive movement events, such as mouse movement events. The PC's keyboard and mouse module can then use a hook to intercept (or block) these events, preventing them from being sent to the PC's Windows system and thus preventing the PC from responding to them. The PC's keyboard and mouse module can also use a hook to capture the operation parameters of the intercepted mouse movement events, such as mouse operation parameters, and send these captured mouse operation parameters to the tablet computer via an established connection through the PC's transmission management module. As an example, when the mouse event is a mouse movement event, the corresponding mouse operation parameters could be: a mouse button flag indicating that the user has moved the mouse, coordinate information indicating the X and Y coordinates of the mouse movement, scroll wheel information (empty), and key information (empty).

[0298] Furthermore, after mouse hover begins, the PC (such as its drag-and-drop management module) can identify the PC's current drag-and-drop status (i.e., whether dragging is in progress). If the PC is currently dragging, it can initiate a continuation of the drag event, or in other words, initiate a cross-screen drag. On the Windows side, since drag-and-drop events need to be initiated and received by a window, the PC can display an invisible window, or a hidden window, after confirming the start of mouse hover. For example, after the PC's keyboard and mouse module confirms the start of mouse hover, it can send a callback indication to the PC's drag-and-drop management module. The PC's drag-and-drop management module can then send a request to the PC's window management module to instruct the creation of a hidden window based on this callback indication. The PC's window management module can then create and display the hidden window based on this request. For example, the PC's window management module can display the hidden window at the edge of the PC screen. The transparency of this hidden window is greater than a threshold, such as being very high or completely transparent.

[0299] After the hidden window is displayed, if the PC is dragging when the mouse hovers over the content, meaning the Windows system initiates a drag event at the time of the mouse hover, the hidden window can receive this drag event from the Windows system. If the PC is not dragging when the mouse hovers over the content, meaning the user moves the mouse without selecting any content, the hidden window will not receive the drag event. After the hidden window receives the drag event, the PC's window management module can obtain the drag event content from it. For example, the PC's window management module can capture the drag event content from the drag event via the DragEnter event. After obtaining the drag event content, the PC's window management module can send it to the tablet computer via the established connection through the PC's transmission management module. Before sending the drag event content to the tablet computer, the PC can also serialize it; that is, the drag event content sent by the PC to the tablet computer can be serialized data.

[0300] For example, such as Figure 17The diagram illustrates a data structure for a drag-and-drop event on a Windows device, as provided in this embodiment. When a drag-and-drop event enters an invisible window, the invisible window receives a data object, such as an IDataObject, corresponding to the drag-and-drop event. The PC's window management module can attach this data object to the IDataObject. Then, the drag-and-drop event content in the IDataObject can be retrieved via the DragEnter event, such as by calling the GetData function. In this embodiment, the drag-and-drop event content required to continue the drag-and-drop event may include text or a file path. As an example, the PC's window management module can retrieve the text in the IDataObject using the GetData(CF_UNICODETEXT) function. The PC's window management module can retrieve the file path in the IDataObject using the GetData(CF_HDROP) function. After obtaining the text or file path, it can be serialized and sent to the tablet computer.

[0301] In addition, in this embodiment, in order to display the dragged object on the tablet computer, such as the animation of the object's shadow moving with the mouse pointer, since shadows and bitmaps can be converted to each other, the PC also needs to obtain the bitmap of the shadow displayed on the PC. For example, the PC can obtain the bitmap of the shadow by capturing an image of the dragged content displayed on the PC screen. For another example, if the dragged content is text, the PC can generate a bitmap of the shadow based on the dragged text. For yet another example, if the dragged content is a file, the PC can find the dragged content based on the obtained file path, determine the type of the dragged content (such as an image file), and then use the corresponding default material as the bitmap of the shadow, or obtain a thumbnail of the content based on the obtained file path of the dragged content as the bitmap of the shadow. The bitmap of the shadow can be sent to the tablet computer through the PC's transmission management module via the established connection with the tablet computer. Before sending the bitmap of the shadow to the tablet computer, the PC can also perform serialization processing on it; that is, the bitmap of the shadow sent by the PC to the tablet computer can be data obtained after serialization processing.

[0302] S1507 The tablet computer receives a bitmap of the drag event content and the shadow, and initiates a drag event based on the bitmap of the drag event content and the shadow.

[0303] S1508: The tablet computer receives mouse operation parameters and simulates mouse movement events based on the mouse operation parameters.

[0304] S1509, The tablet computer generates a mouse press event.

[0305] After the tablet receives the drag event content and the shadow bitmap, it can parse them and initiate the drag event. For the Android system, since drag events need to be initiated and received by view controls, the tablet can open a transparent activity, or an invisible activity. This invisible activity has view controls with transparency greater than a threshold. Using this view control, by calling the Android Open Source Project (AOSP) interface, the tablet can initiate the corresponding drag event based on the received drag event content and shadow bitmap, thus enabling the continuation of drag events on the tablet.

[0306] For example, in combination Figure 2B ,and Figure 17 In the example shown, after the tablet's transfer management module receives the drag event content and the bitmap of the shadow, it can transmit these to the tablet's drag management module. The tablet's drag management module parses the received drag event content to obtain the text or file path from the PC. Based on the obtained text or file path, the PC's drag management module can construct the drag event content data (clipData). Additionally, the tablet's drag management module can generate the corresponding shadow based on the received shadow bitmap. Then, the tablet's drag management module can use the view control of the transparent activity opened on the tablet to call the startDragAndDrop method provided by the AOSP interface, using clipData and the shadow as input parameters, to initiate the drag event on the tablet. Of course, if the PC serializes the drag event content and shadow bitmap before sending it to the tablet, the tablet can deserialize the received data to obtain the drag event content and shadow bitmap.

[0307] For example, Figure 18 The diagram shown is a data structure diagram of a drag event (DragEvent) on an Android device according to an embodiment of this application. Combined with... Figure 18The tablet's drag-and-drop management module can construct content data (clipData) based on the text or file path (uri) received from the PC (where the text or file path is contained in the content items included in the content data), and generate the corresponding shadow based on the received shadow bitmap. Then, it calls the AOSP's startDragAndDrop method, taking the clipData, shadow, and other parameters obtained based on the user's mouse operation information (such as received mouse movement events) as input parameters to initiate (dispatch) the drag-and-drop event (DragEvent) on the tablet.

[0308] It should be noted that, in combination Figure 18 The content description includes a label. This label indicates whether the drag-and-drop event was initiated by the tablet's drag-and-drop management module or by an application on the tablet. For example, the label is a string. For instance, when the label is "windowscast", it indicates that the drag-and-drop event was initiated by the tablet's drag-and-drop management module; when the label is not "windowscast", it indicates that the drag-and-drop event was initiated by an application on the tablet, not the drag-and-drop management module. In the example above, since the drag-and-drop event was initiated by the drag-and-drop management module, the label in the drag-and-drop event's content description is "windowscast". The specific descriptions and construction rules of other parameters are similar to the corresponding implementations of generating raw drag-and-drop events on Android in existing technologies, and will not be repeated here.

[0309] It is understandable that drag-and-drop events can be triggered by drag-and-drop operations, which can include press operations (such as mouse press operations) and move operations (such as mouse move operations). When a user wants to transfer content from the PC to the tablet via drag-and-drop and continue dragging on the tablet, after the user moves the mouse to trigger mouse navigation, the user will continue to move the mouse in the same direction. That is, the user will use the PC input device to input a move operation, such as using the PC mouse to input a mouse move operation. At this time, as described in S1505, after the user inputs a mouse move operation using the PC mouse, the PC's keyboard and mouse module will intercept the corresponding received move event, such as a mouse move event, and capture and process the operation parameters contained in the mouse move event, such as mouse operation parameters, and send them to the tablet through the PC's transmission management module, such as the tablet's transmission management module. At this time, the tablet's transmission management module can receive the mouse operation parameters. Since the operating systems of the PC and the tablet are different, the key values ​​of the mouse operation parameters in their mouse events are different. Therefore, after receiving the mouse operation parameters, the tablet computer can convert the key codes of the received mouse operation parameters into key codes that the tablet computer can recognize, according to a preset mapping relationship. Then, the tablet computer (such as its keyboard and mouse module) can use the created virtual input device to simulate input events that the tablet computer can recognize, such as mouse events, based on the converted key codes of the mouse operation parameters. Specifically, it can simulate movement events that the tablet computer can recognize, such as mouse movement events. The tablet computer's keyboard and mouse module can also send mouse movement events to the tablet computer's framework layer.

[0310] Furthermore, at the start of mouse navigation, the initiating end, i.e., the PC, is in a drag-and-drop state, meaning the PC is in a mouse-pressed state. The target end, i.e., the tablet, can only receive mouse movement events at this time, meaning it is not in a mouse-pressed state. Therefore, the tablet can generate a press operation, such as a mouse press operation. For example, after the tablet calls the `startDragAndDrop` method provided by the AOSP interface, it receives a drag-and-drop start callback (`onDragStart`). At this point, the tablet can determine whether the drag-and-drop event was initiated by its drag-and-drop management module based on the label in the drag-and-drop event. When it is determined that the drag-and-drop event was initiated by the tablet's drag-and-drop management module (e.g., label "windowscast"), the tablet can generate (or simulate) a press event, such as a mouse press event, using the created virtual input device, based on the operation parameters of the press operation, such as a mouse press operation. For example, the tablet's drag-and-drop management module controls the tablet's keyboard and mouse module to send a mouse press event to the tablet's frame layer using the created virtual input device. In this way, the drag-and-drop event initiated in S1507 on the tablet can be attached to the mouse pointer displayed on the tablet's screen.

[0311] S1510, the tablet computer executes drag events based on mouse movement events and mouse press events, and displays an animation of the shadow of the content moving with the mouse pointer on the tablet computer's screen.

[0312] The tablet computer (such as the tablet's frame layer) can execute the drag event initiated in S1507 based on movement events, such as mouse movement events and press events, such as mouse press events. During the execution of the drag event, the tablet computer can also display content on its screen, such as an animation of the content's shadow moving with the mouse pointer (which can be the first cursor in this application). For example, combined with... Figure 16 ,like Figure 19 As shown, as the mouse 1901 moves, the shadow 1903 of the corresponding image displayed on the tablet computer's screen 1902 moves with the mouse pointer 1904 in an animation. Figure 19 The shadow 1903 in the image shown is dragged along the mouse pointer 1904 as shown in trajectory 1905.

[0313] After a user drags and drops content from a PC to a tablet and continues dragging on the tablet, the drag event is attached to the mouse pointer displayed on the tablet screen. Therefore, the user can precisely select the release point using the mouse pointer. For example, if the user wants to use or manipulate the content on the tablet, they can move the mouse pointer over the desired view control on the tablet and then release the mouse. After the mouse is released, the PC's keyboard and mouse module receives the corresponding release event, such as a mouse lift-off event. Since this mouse lift-off event is received by the PC after mouse hovering begins, the PC's keyboard and mouse module uses a hook to intercept (or block) it, preventing the mouse lift-off event from being sent to the PC's Windows system, thus preventing the PC from responding to the received mouse lift-off event. The PC's keyboard and mouse module can also use a hook to capture the operation parameters contained in the mouse lift-off event, such as mouse operation parameters, and send the captured mouse operation parameters to the tablet via the PC's transmission management module through an established connection. The mouse operation parameters for the mouse release event can include: a mouse button flag indicating that the user has released the mouse, coordinate information (empty), scroll wheel information (empty), and key information indicating that the user has operated the left mouse button. Correspondingly, the tablet's transmission management module can receive these mouse operation parameters. After receiving the mouse operation parameters, the tablet can convert the key codes of the received mouse operation parameters into key codes that the tablet can recognize, according to a preset mapping relationship. Then, the tablet uses the created virtual input device to simulate mouse events that the tablet can recognize, such as the mouse operation parameters, based on the converted key codes. This allows it to simulate the corresponding input event, such as the mouse release event.

[0314] After simulating a mouse release event, the tablet can determine the mouse release point based on the current coordinates of the mouse pointer. For example, after the tablet detects the start of keyboard and mouse movement, it can register a listener for the mouse pointer's coordinates. This listener allows the tablet to monitor the mouse pointer's coordinates on the screen in real time. Based on this, upon receiving a mouse release event, the tablet can obtain the current coordinates of the mouse pointer on the screen using this listener. Based on these coordinates, the tablet can determine the mouse release point. As another example, after the tablet calls the AOSP's `startDragAndDrop` method to initiate a drag event, it monitors input events. For instance, when the user continues dragging on the tablet, it can detect movement events, such as mouse movement events. Based on these mouse movement events, the tablet can obtain the operation parameters of the mouse movement event, such as mouse operation parameters, which can be extracted from `MotionEvent`. These mouse operation parameters include coordinate information indicating the mouse position. Then, when the user lifts their finger, the tablet detects the lift event, such as a mouse lift event. Based on the previously obtained coordinate information, the tablet can determine the coordinate position of the mouse pointer, and thus determine the mouse release point based on the obtained coordinate position.

[0315] After the user releases the mouse, the PC directly intercepts and processes the received release event, such as the mouse release event. Therefore, the drag event on the PC is not released. At this time, the tablet can send a signal to the PC through the established connection after receiving the mouse release event to inform the PC to release the drag event.

[0316] In addition, the tablet computer can also respond to release events, such as mouse release events. For example, taking the content in S1502 as text, after simulating a mouse release event, the tablet computer's drag management module can send the content data of the aforementioned drag event to the view control at the mouse release point. This content data includes the text. After receiving the content data, the view control can process it accordingly, such as displaying the text in the view control. Another example is taking the content in S1502 as a file. Since the file is not actually transferred to the tablet computer during the drag-and-drop process, after the user releases the mouse and the PC receives the aforementioned signal from the tablet computer informing it of the release drag event, the PC can transfer the file to the tablet computer. After receiving the file, the tablet computer can store it in a predetermined cache directory. At this time, the tablet computer's drag management module can obtain the file's URI (e.g., URI 1). This URI 1 is the path of the file in the cache directory, which is different from the URI in the drag event content sent by the PC to the tablet computer in S1507 (the URI sent by the PC is the storage path of the file on the PC). The tablet's drag-and-drop management module constructs new content data based on URI 1 and sends this content data to the view control at the mouse release point as a response to a mouse lift-off event. Upon receiving the content data, the view control can perform corresponding processing. For example, if the view control is a view control in a memo, it can display the file; if the view control is an input box in a chat window, it can send the file out.

[0317] In some other embodiments, when the tablet is also connected to other devices, such as a mobile phone, if the user wants to continue dragging content to the phone, they can continue moving the mouse so that the mouse pointer on the tablet slides across the edge of the tablet screen, triggering the mouse to move from the tablet to the phone, thus continuing the drag-and-drop event on the phone. It should be noted that after the mouse moves to the phone, the PC can directly interact with the phone to allow the phone to continue the drag-and-drop event. The specific description of how drag-and-drop events are continued on the phone is similar to that on the tablet, and will not be repeated here.

[0318] Understandably, through the above process, without enabling screen mirroring, users can use an input device, such as a PC mouse, to drag and drop text or files from PC applications to the edge of the PC screen and continue moving in the same direction, triggering mouse hovering. After mouse hovering begins, the mouse pointer appears on the tablet. Furthermore, by sending the drag event content to the tablet, the PC can attach the drag event to the tablet's mouse pointer, thus continuing the drag event on the tablet and giving the user the visual effect of content being dragged from the PC to the tablet.

[0319] In this embodiment, the user may not only need to transfer content from the PC to the tablet via drag and drop, and continue dragging on the tablet, but also may need to transfer content from the tablet to the PC via drag and drop. For example, referring to the examples in S1501-S1510 above, the user transfers image 1 from the PC to the tablet via drag and drop, continues dragging on the tablet, and then releases the mouse. Afterwards, the user edits image 1 using the tablet's stylus and saves it. The user wants to transfer the edited image 1 to the PC via drag and drop, continue dragging on the PC, and then release the mouse to save the edited image 1 on the PC.

[0320] Please refer to Figure 20 This is a flowchart illustrating another cross-device object dragging method provided in an embodiment of this application. The following is combined with... Figure 15 The illustrated embodiment takes as an example a user wants to transfer content from a tablet to a PC via drag-and-drop after mouse hovering, and then continue dragging and dropping on the PC. This example illustrates the method provided in this embodiment. Figure 20 As shown, the method may include the following S2001-S2010.

[0321] S2001, The tablet computer receives mouse operations on selected content from the PC.

[0322] S2002. The tablet computer receives mouse operation parameters from the PC, including mouse press events and mouse move events. Based on the mouse operation parameters, it simulates mouse press events and mouse move events, initiates drag events based on the mouse press events and mouse move events, and displays an animation of the shadow of the content moving with the mouse pointer on the tablet computer's screen.

[0323] In this embodiment, the mouse operation parameters for the mouse press event can be the first operation parameters in this application, and the mouse operation parameters for the mouse move event can be the second operation parameters in this application.

[0324] like Figure 15As described in the illustrated embodiment, after the keyboard and mouse sharing mode is enabled, if mouse hovering occurs, the PC will use a hook to intercept the corresponding input event, such as a mouse event, after the user operates the PC's input device, such as the mouse. The PC will then capture the operation parameters contained in the intercepted mouse event, such as mouse operation parameters, and send them to the tablet computer. For example, consider a scenario where, after the mouse hovers over the tablet computer, the user wants to transfer content from the tablet computer to the PC via drag and drop, and continue dragging and dropping on the PC. Combined with... Figure 2B The user uses the PC's input devices, such as a mouse, to select and drag content. The PC's keyboard and mouse module receives the corresponding input operation, such as a mouse operation. The PC's keyboard and mouse module uses hooks to intercept the received input events, such as mouse events, so that the input event is not sent to the PC's Windows system, thus preventing the PC from responding to the received input event. The PC's keyboard and mouse module also uses hooks to capture the operation parameters in the intercepted input events, such as mouse operation parameters, and transmits them to the tablet computer through the PC's transmission management module via an established connection. At this time, the tablet computer's transmission management module can receive the corresponding operation parameters, such as mouse operation parameters. After converting the received operation parameters into key codes, the tablet computer can simulate the corresponding input event, such as a mouse event, using the created virtual input device.

[0325] Then, when the tablet's mouse pointer is displayed on the object to be dragged, such as the content selected by the user, the user can use the PC's input device, such as the mouse, to input a drag operation. The tablet can then drag the corresponding object, i.e., the content, based on this drag operation. This drag operation can be an action that indicates the initiation of a drag event on the selected content. The drag operation can include one or more actions. For example, a drag operation might include two actions: a press operation and a move operation. Taking a mouse as the input device, the press operation could be a mouse press operation, and the move operation could be a mouse move operation. For instance, the user can press and move the PC's mouse, thus inputting both mouse press and move operations using the PC's mouse. The PC's keyboard and mouse module can then receive the press event (e.g., mouse press event) and move event (e.g., mouse move event). Similarly, the PC's keyboard and mouse module can use a hook to intercept the received mouse press and move events, and then use the hook to capture the operation parameters of the intercepted mouse press and move events, such as mouse operation parameters, and transmit them to the tablet via the established connection through the PC's transmission management module. At this point, the tablet's transmission management module can receive the corresponding operation parameters, such as mouse operation parameters. After converting the received operation parameters into key codes, the tablet can simulate corresponding input events using the created virtual input device, such as simulating press events (e.g., mouse press events) and movement events (e.g., mouse move events).

[0326] After the tablet computer simulates press events (such as mouse press events) and movement events (such as mouse move events), in response to these events, the tablet computer can initiate drag events through corresponding applications (e.g., if the selected content is a Word document, the application is an office application; if the selected content is an image in a file manager, the application is a file manager; if the selected content is a piece of text in a memo, the application is a memo) and draw the content, such as the animation of the content's shadow following the mouse pointer movement, displayed on the tablet computer's screen. For example, a user edits and saves image 1 dragged from a PC using the tablet computer's stylus. The user wants to drag the edited image back to the PC and continue dragging within the PC. Figure 21 As shown, the user can select the edited image 2102 using mouse 2101, and then press and move mouse 2101. As mouse 2101 moves, the tablet computer displays an animation on its screen 2103 of the shadow 2106 of the edited image 2102 moving with the mouse pointer 2104, as shown. Figure 21 The shadow 2106 of the image 2102 shown in the figure follows the dragging trajectory of the mouse pointer 2104 as shown in trajectory 2105.

[0327] It should be noted that the above explanation uses an example where drag-and-drop events include both press and move events. For instance, a user can trigger drag-and-drop by pressing and moving the mouse. In some other embodiments, the events triggering drag-and-drop may only include press events. For example, a user can trigger drag-and-drop by long-pressing the mouse, or by pressing the mouse button. This embodiment does not impose any specific limitations.

[0328] S2003: The tablet computer obtains the bitmap of the drag event content and shadow, and caches the obtained bitmap of the drag event content and shadow.

[0329] The drag-and-drop event content is used to construct drag-and-drop events for the connecting device. For example, when the content in S2001 is text, the drag-and-drop event content can include text. As another example, when the content in S2001 is a file or folder, the drag-and-drop event content can include the file path (e.g., URI).

[0330] For example, in combination Figure 2B When a tablet application initiates a drag-and-drop event, the corresponding drag-and-drop event content can be called back from the framework layer to the application layer. For example, when the application calls the `startDragAndDrop` method provided by the AOSP interface to initiate a drag-and-drop event, the framework layer can extract the drag-and-drop event content using instrumentation. The instrumentation locations and content can be predetermined, and then instrumentation can be performed based on these locations and content to extract the drag-and-drop event content. Afterward, the tablet's framework layer can call back the extracted drag-and-drop event content to the drag-and-drop management module of the tablet's application layer. The tablet's drag-and-drop management module can then cache this drag-and-drop event content. As an example, the drag-and-drop event content required to continue the drag-and-drop event can include text or a file path. When the application initiates a drag-and-drop event, the framework layer can extract the text or file path using instrumentation.

[0331] Additionally, in this embodiment, drag events may be initiated in two ways: one is initiated by the tablet's drag management module (i.e., the drag is performed from the PC to the tablet, such as...). Figure 15 (As described in the illustrated embodiment), another type is initiated by a tablet application. And in conjunction with... Figure 15 In the illustrated embodiment, Figure 18As described in the data structure, in this embodiment, the content description of a drag-and-drop event on the tablet includes a label indicating whether the drag-and-drop event was initiated by the tablet's drag-and-drop management module or by an application on the tablet. It's understood that the drag-and-drop event content only needs to be retrieved and cached when it's initiated by an application on the tablet, or not by the tablet's drag-and-drop management module, so that it can be used by the PC to continue the drag-and-drop event after it's transferred to the PC. In other words, after a drag-and-drop event is initiated, the tablet can distinguish whether the event was initiated by an application on the tablet or by the tablet's drag-and-drop management module based on the label in the drag-and-drop event content description. For example, if the label is not "windowscast", it can be determined that the drag-and-drop event was initiated by an application on the tablet, not by the tablet's drag-and-drop management module. Only when the drag-and-drop event is not initiated by the tablet's drag-and-drop management module does the framework layer need to retrieve the drag-and-drop event content and send it to the tablet's drag-and-drop management module so that it can cache the drag-and-drop event content.

[0332] Additionally, in this embodiment, the framework layer can also obtain the shadow bitmap by adding a new interface or utilizing existing interfaces (such as the interface that calls back clipData). The obtained shadow bitmap can also be sent back to the drag-and-drop management module of the tablet application layer. After obtaining the shadow bitmap, the tablet's drag-and-drop management module can also cache it.

[0333] S2004. The tablet computer monitors the coordinates of the mouse pointer on the tablet computer screen.

[0334] S2005. Based on the coordinates of the mouse pointer on the tablet computer screen, when the tablet computer determines that the mouse pointer has moved out of the edge of the tablet computer screen, it sends a shuttle status information to the PC to indicate that the mouse shuttle has ended.

[0335] S2006, The PC receives shuttle status information used to indicate the end of mouse shuttle.

[0336] The shuttle status information used to indicate the end of mouse shuttle can be the second shuttle status information in this application.

[0337] As content (such as its shadow) moves with the mouse pointer, the tablet computer can determine whether the dragged content (such as its shadow) has been dragged off the edge of the tablet screen. When the content (such as its shadow) is dragged off the edge of the tablet screen, it indicates that the user wants to use the mouse to control other devices. In this embodiment, the specific description of content being dragged off the edge of the tablet screen is similar to the specific description of content being dragged off the PC, and will not be repeated in detail here. The following explanation uses the determination of whether the dragged content (such as its shadow) has been dragged off the edge of the tablet screen as an example of determining whether the mouse pointer has slipped off the edge of the tablet screen.

[0338] As an example, after a drag event is initiated, the mouse pointer moves on the tablet's screen, and the tablet can monitor the real-time coordinates of the mouse pointer on the screen. For instance, after mouse and keyboard movement begins, the tablet can register a listener for the mouse pointer's coordinates. This listener allows the tablet to monitor the mouse pointer's coordinates on the screen in real time. Based on the real-time coordinates detected by the listener, the tablet can determine whether the mouse pointer has crossed the edge of the screen. For example, the tablet can determine the mouse pointer's coordinates based on its initial position and relative displacement, thus determining whether the mouse pointer has crossed the edge of the screen. The initial position of the mouse pointer can be its coordinates on the screen when the mouse begins to move, or its coordinates before the mouse begins to move. Specifically, this initial position can refer to the coordinates in a coordinate system with the top-left corner of the screen as the origin, the X-axis pointing from the top-left corner to the right edge of the screen, and the Y-axis pointing from the top-left corner to the bottom edge of the screen. The specific implementation of determining whether the mouse pointer moves off the edge of the tablet computer screen is similar to that of determining whether the mouse pointer moves off the edge of the PC screen, and will not be described in detail here.

[0339] After the tablet computer determines that the mouse pointer has moved across the edge of the tablet screen, it indicates that the user wants to use the mouse to control other devices. As described in S1506, if the tablet computer is connected to only one device, the user wants to use the mouse to control the PC. If the tablet computer is connected to multiple devices, the tablet computer can display a list of options, including the identifiers of all devices connected to the tablet computer, for the user to select the device they want to control with the mouse. If the user selects the PC identifier, it indicates that the user wants to use the mouse to control the PC. Alternatively, a shuttle relationship can be pre-configured in the tablet computer to determine which device the mouse shuttles to, i.e., which device will respond to the mouse operation. The specific description of the configuration and application of the shuttle relationship is similar to the corresponding content in the above embodiments and will not be repeated in detail here. When it is determined that the user wants to use the mouse to control the PC, the tablet computer can determine that the mouse shuttle has ended. At this time, the tablet computer can send shuttle status information to the PC through the established connection to indicate that the mouse shuttle has ended.

[0340] S2007. The PC determines the end of mouse movement based on the received movement status information.

[0341] S2008: The PC cancels the interception of mouse events and generates mouse press events.

[0342] S2009: The PC obtains a bitmap of the drag event content and shadow from the tablet computer, and initiates a drag event based on the bitmap of the drag event content and shadow.

[0343] The drag data in this application may include the drag event content and a bitmap of the shadow. After the PC receives the navigation status information indicating the end of mouse navigation, it can determine that the mouse navigation has ended.

[0344] After the mouse hovering is complete, the PC can display the mouse pointer on the PC screen. Combined with... Figure 15 As described in the illustrated embodiment, when mouse hovering begins, the PC hides the mouse pointer on the PC screen. Therefore, after mouse hovering is determined to be over, the PC can redisplay the mouse pointer on the PC screen. Furthermore, the trigger condition for starting mouse hovering is that the mouse pointer moves across the edge of the PC screen; therefore, the mouse pointer is displayed at the edge of the PC screen before being hidden. In this embodiment, when mouse hovering ends, the PC unhides the mouse pointer, and the mouse pointer will reappear at the edge of the PC screen. Of course, when mouse hovering ends, the mouse pointer on the tablet will also not be displayed. This provides the user with the visual effect of the mouse pointer hovering from the tablet to the PC.

[0345] After the mouse hovering event is confirmed to be complete, the PC needs to uninstall (or disable) the hook to remove the interception of input device events, such as mouse events. For example, after a user drags and drops content from a tablet to the PC, triggering the end of the mouse hovering event, the user will continue to move the mouse in the same direction. The PC's keyboard and mouse module can receive movement events, such as mouse movement events. Since the hook has been uninstalled at this point, the PC's keyboard and mouse module will send the received movement events, such as mouse movement events, to the PC's Windows system so that the Windows system can respond to the movement events.

[0346] Furthermore, before the mouse hover ends, the tablet is in a drag-and-drop state. The mouse hovering back to the PC is to continue the drag-and-drop process, meaning the drag-and-drop needs to continue on the PC. On the Windows platform, the mouse must be pressed on a window to initiate drag-and-drop correctly. However, while the user continues to move the mouse, the PC can only receive movement events, such as mouse move events, and not press events, such as mouse press events. Therefore, the PC (such as the PC's drag-and-drop management module) can generate a press event, such as a mouse press event, and transmit it to an invisible window. For example, after the PC determines that the mouse hovering has ended and the tablet is still dragging, the PC can generate a mouse press event and transmit it to an invisible window so that the drag-and-drop event initiated on the PC's S2009 can be attached to the mouse pointer displayed on the PC screen, thus continuing the drag-and-drop event on the PC.

[0347] In this embodiment, after determining that the mouse hovering has ended, the PC can also request the tablet computer's drag status (i.e., whether it is currently dragging). When the tablet computer returns a drag status indicating that it is currently dragging, the PC can request the drag event content and a bitmap of the shadow from the tablet computer. The PC can send a request message to the tablet computer, which can be used to request drag data, i.e., to request the drag event content and the bitmap of the shadow.

[0348] For example, in combination Figure 2BWhen the PC determines that mouse hovering has ended, the PC's keyboard and mouse module can send an indication that mouse hovering has ended to the PC's drag-and-drop management module. Based on the received indication, the PC's drag-and-drop management module can request the drag status from the tablet computer via the PC's transmission management module. When the tablet computer returns a drag status indicating that dragging is in progress, the PC's drag-and-drop management module can request the drag event content and shadow bitmap from the tablet computer via the PC's transmission management module. Correspondingly, the tablet computer's transmission management module can receive this request and forward it to its drag-and-drop management module. Upon receiving this request, the tablet computer's drag-and-drop management module can feed back the drag event content and shadow bitmap cached in S2003 to the PC's transmission management module via the tablet computer's transmission management module.

[0349] Based on the description in S2003, after receiving the drag-and-drop event content, the PC's transmission management module can transmit the drag-and-drop event content to its own drag-and-drop management module. The PC's drag-and-drop management module parses the received drag-and-drop event content to obtain the text or file path from the tablet. Based on the obtained text or file path, the PC's drag-and-drop management module can construct a drag-and-drop event data object, such as an IDataObject. Additionally, after receiving the shadow bitmap, the PC's transmission management module can restore the shadow on the PC based on the bitmap. For example, the shadow can be restored using the IDragSourceHelper interface provided by the PC. Afterward, the PC can initiate drag-and-drop events on the PC side. For example, combined with... Figure 15 As described in the illustrated embodiment, when mouse hover begins, the PC displays an invisible window. Therefore, the PC can initiate the drag event using this invisible window. The invisible window can be closed after the drag event is initiated.

[0350] S2010: The PC executes drag events based on mouse movement and mouse press events, and displays an animation of the content's shadow moving with the mouse pointer on the PC's screen.

[0351] After a drag event is initiated, the PC can execute the drag event in response to movement events, such as mouse move events, and press events, such as mouse press events. Content will then be displayed on the PC's screen, such as an animation of the content's shadow moving with the mouse pointer. For example, combined with... Figure 21 ,like Figure 22 As shown, as the mouse 2201 moves, the PC displays an animation on the PC screen 2202 of the shadow 2203 of the edited image, showing the movement of the mouse pointer 2204. Figure 22 The dragging trajectory of the shadow 2203 as the mouse pointer 2204 moves is shown in trajectory 2205.

[0352] It should be noted that the above example illustrates the scenario where the user first drags and drops content from the PC to the tablet, and then drags and drops content from the tablet to the PC. In other embodiments, the user may not need to drag and drop content from the PC to the tablet; instead, they can directly drag and drop content from the tablet to the PC. The specific implementation of this embodiment is different from... Figure 15 and Figure 20 The specific implementation of the illustrated embodiment is similar, except that S1502 is not executed. S1503 is replaced by receiving a movement event (such as a mouse movement event) and displaying an animation of mouse pointer movement on the PC's display screen based on the movement event. S1506 is not executed, but the operation of displaying an invisible window on the PC is still required; however, the invisible window will not receive drag events. S1507 and S1509 are not executed. S1510 is replaced by the tablet computer displaying an animation of mouse pointer movement on its display screen based on a movement event, such as a mouse movement event. That is, after mouse hovering begins, the tablet computer can receive the user's operation of moving the mouse pointer (such as referred to as the first cursor) displayed on the tablet computer's input device, such as a mouse. In response to this operation, the tablet computer can display an animation of mouse pointer movement on its display screen. If the user wants to drag some content, they can continue to move the PC's mouse until the mouse pointer moves onto that content. In this embodiment, the tablet computer receives input from the user using a PC's input device, such as a mouse, indicating movement of the first cursor. Specifically, the PC intercepts the corresponding movement event, such as a mouse movement event, and sends the operation parameters included in the movement event, referred to as third operation parameters, to the tablet computer. The tablet computer can then simulate the movement event, such as a mouse movement event, based on these third operation parameters, thereby displaying an animation of mouse pointer movement on the tablet computer's screen. Other operations are similar, and will not be described in detail here.

[0353] The above embodiment illustrates how the tablet caches drag data at the start of the drag-and-drop operation, and the PC requests the drag data from the tablet after the mouse hovering ends. In other embodiments, the tablet may not cache drag data at the start of the drag-and-drop operation; instead, it may retrieve the drag data and have the PC actively send it after the mouse hovering ends, without requiring a PC request.

[0354] The method provided in this embodiment, without initiating screen mirroring, utilizes keyboard and mouse sharing technology, allowing users to drag and drop text or files between multiple terminals participating in collaborative work. Users can also process the transmitted content using these terminals, ensuring that the hardware capabilities of all terminals can participate in collaborative work. Furthermore, since screen mirroring is not required, it does not occupy display space on any single terminal. This improves the efficiency of multi-terminal collaborative use and enhances the user experience.

[0355] Furthermore, the above embodiments are illustrated using a mouse as the input device. In this embodiment, the input device can also be a touchpad. When the input device is a touchpad, the user can use the touchpad buttons (left or right button) to input press operations and slide their finger on the touchpad to input movement operations. The specific implementation of dragging objects using a touchpad is similar to that using a mouse, and will not be described in detail here.

[0356] Figure 23 This is a schematic diagram illustrating the composition of a cross-device object dragging device provided in an embodiment of this application. Figure 23 As shown, the device can be applied to a second terminal (such as the aforementioned mobile phone or tablet computer), which is connected to the first terminal. The device may include a display unit 2301, a receiving unit 2302, and a sending unit 2303.

[0357] The display unit 2301 is used to display the first cursor on the object displayed on the second terminal.

[0358] The receiving unit 2302 is used to receive drag operations input by the user using the input device of the first terminal, and the drag operations are used to initiate drag operations on an object.

[0359] The display unit 2301 is also used to display an animation of an object moving with the first cursor on the display screen of the second terminal in response to a drag operation.

[0360] The sending unit 2303 is used to send drag data to the first terminal when it is determined that the object has been dragged out of the edge of the display screen of the second terminal.

[0361] Furthermore, the aforementioned drag data can be used by the first terminal to display an animation of an object moving with the second cursor on the first terminal's display screen.

[0362] Furthermore, the object can be an application icon; or, the object can be a window that includes the application's interface; when the object is an application icon, the dragged data includes the application icon; when the object is a window, the dragged data includes the application's interface.

[0363] Furthermore, the receiving unit 2302 is also used to receive an indication message from the first terminal. The indication message is sent by the first terminal after the user inputs a drag-and-release operation using the input device of the first terminal, and is used to indicate that the dragging of the object has ended.

[0364] The sending unit 2303 is also used to send data to the first terminal, the data being used by the first terminal to display the application interface on the display screen of the first terminal.

[0365] Furthermore, when the object is an application icon, the device may also include a creation unit 2304.

[0366] Create unit 2304 to create a drag-and-drop object based on the application's icon. The drag-and-drop object includes an icon that is the same as the application's icon.

[0367] Display unit 2301 is specifically used to respond to drag operations and display an animation of the dragged object moving with the first cursor.

[0368] Furthermore, when the object is a window, the display unit 2301 is also used to display the application's interface; the receiving unit 2302 is also used to receive a long press operation input by the user using the input device of the first terminal when the first cursor is displayed on the status bar of the second terminal, for triggering the application's interface windowing; the display unit 2301 is also used to display a window in response to the long press operation; the window specifically includes a screenshot of the application's interface, and the dragged data specifically includes a screenshot of the application's interface.

[0369] Furthermore, the display unit 2301 is specifically used to display an animation of the shadow of an object moving with the first cursor on the display screen of the second terminal.

[0370] Furthermore, the object is text, a file, or a folder; the drag data includes the drag event content and a bitmap of the shadow; where, when the object is text, the drag event content includes the text, and when the object is a file or folder, the drag event content is the file path.

[0371] Furthermore, the device may also include an acquisition unit 2305 for acquiring and caching dragged data.

[0372] Furthermore, the receiving unit 2302 is also used to receive a request message from the first terminal, the request message being used to request drag data; the sending unit 2303 is specifically used to send the cached drag data to the first terminal.

[0373] Furthermore, the receiving unit 2302 is also used to receive first shuttle status information from the first terminal, the first shuttle status information being used to indicate the start of shuttle.

[0374] The display unit 2301 is also used to display the first cursor on the second terminal in response to the first shuttle status information.

[0375] Furthermore, the receiving unit 2302 is also used to receive the operation of moving the first cursor input by the user using the input device of the first terminal.

[0376] The display unit 2301 is also configured to, in response to the operation of moving the first cursor, display an animation of the first cursor moving on the display screen of the second terminal until the first cursor moves onto the object.

[0377] Furthermore, the drag operation includes a press operation and a move operation; the receiving unit 2302 is specifically used to receive a first operation parameter from the first terminal, the first operation parameter being the operation parameter contained in the press event received by the first terminal after the user performs a press operation using the input device of the first terminal; and to receive a second operation parameter from the first terminal, the second operation parameter being the operation parameter contained in the move event received by the first terminal after the user performs a move operation using the input device of the first terminal.

[0378] The device may further include: a simulation unit 2306, used to simulate a press event according to a first operation parameter and a move event according to a second operation parameter, wherein the press event and the move event are used to indicate the initiation of a drag-and-drop operation on an object.

[0379] Display unit 2301 is specifically used to display an animation of an object moving with a first cursor in response to a press event and a move event.

[0380] Furthermore, the receiving unit 2302 is specifically used to receive a third operation parameter from the first terminal. The third operation parameter is the operation parameter contained in the movement event received by the first terminal after the user performs a movement operation using the input device of the first terminal.

[0381] The simulation unit 2306 is used to simulate a movement event based on a third operation parameter. The movement event is used to indicate the movement of the first cursor.

[0382] Furthermore, the sending unit 2303 is also used to send second shuttle status information to the first terminal, the second shuttle status information being used to indicate the end of the shuttle.

[0383] Furthermore, the long press operation includes a press operation and a release operation; the receiving unit 2302 is specifically used to receive a fourth operation parameter from the first terminal, the fourth operation parameter being the operation parameter contained in the press event received by the first terminal after the user performs a press operation using the input device of the first terminal when the first cursor is displayed on the status bar; and to receive a fifth operation parameter from the first terminal, the fifth operation parameter being the operation parameter contained in the release event received by the first terminal after the user performs a press operation for a period of time and then performs a release operation using the input device of the first terminal.

[0384] The simulation unit 2305 is used to simulate a press event according to the fourth operation parameter and a release event according to the fifth operation parameter. The press event and the release event are used to trigger the windowing of the application interface.

[0385] Display unit 2301 is specifically used to display a window in response to press and release events.

[0386] Furthermore, the creation unit 2304 is also used to create a virtual input device after the connection with the first terminal is successfully established; or, the receiving unit 2302 is also used to receive a notification message from the first terminal, the notification message indicating that the keyboard and mouse sharing mode of the first terminal has been enabled, and the creation unit 2304 is also used to create a virtual input device in response to the notification message; wherein, the virtual input device is used by the second terminal to simulate input events according to the operation parameters.

[0387] Figure 24 This is a schematic diagram illustrating the composition of another cross-device object dragging device provided in an embodiment of this application. (See attached diagram.) Figure 24 As shown, the device can be applied to a first terminal (such as the PC mentioned above), the first terminal is connected to a second terminal, and the device may include a transmitting unit 2401 and a receiving unit 2402.

[0388] The sending unit 2401 is used to send data of a drag operation input by a user using the input device of the first terminal to the second terminal. The drag operation is used to initiate dragging of an object displayed on the second terminal.

[0389] The receiving unit 2402 is used to receive drag data from the second terminal. The drag data is sent to the first terminal after the second terminal determines that the object has been dragged out of the edge of the second terminal's display screen during the process of the object moving on the display screen of the second terminal with the first cursor.

[0390] Furthermore, the device may also include a display unit 2403 for displaying an animation of an object moving with a second cursor on a display screen of the first terminal based on drag data and drag operations.

[0391] Furthermore, the object can be an application icon; or, the object can be a window that includes the application's interface; when the object is an application icon, the dragged data includes the application icon; when the object is a window, the dragged data includes the application's interface.

[0392] Furthermore, the receiving unit 2402 is also used to receive drag-and-release operations input by the user using the input device of the first terminal.

[0393] The display unit 2403 is also used to display the application interface on the display screen of the first terminal in response to a drag-and-release operation.

[0394] Furthermore, the sending unit 2401 is also used to send an indication message to the second terminal, the indication message being used to indicate the end of dragging the object.

[0395] The receiving unit 2402 is also used to receive data from the second terminal.

[0396] Display unit 2403 is specifically used to display the application interface on the display screen of the first terminal based on data.

[0397] Furthermore, when the object is a window, the sending unit 2401 is also used to send data of a long press operation input by the user using the input device of the first terminal to the second terminal. The long press operation is input by the user when the first cursor is displayed on the status bar of the second terminal, and is used to trigger the windowing of the application interface displayed on the second terminal. The window specifically includes a screenshot of the application interface, and the dragged data specifically includes a screenshot of the application interface.

[0398] Furthermore, the display unit 2403 is specifically used to display an animation of the shadow of an object moving with the second cursor on the display screen of the first terminal, based on the drag data and drag operation.

[0399] Furthermore, the object is text, a file, or a folder; the drag data includes the drag event content and a bitmap of the shadow; where, when the object is text, the drag event content includes the text, and when the object is a file or folder, the drag event content is the file path.

[0400] Furthermore, the device may also include: a determining unit 2404 for determining that the second cursor slides out of the edge of the display screen of the first terminal.

[0401] Furthermore, the receiving unit 2402 is also used to receive the operation of moving the second cursor input by the user using the input device of the first terminal.

[0402] The display unit 2403 is also used to display an animation of the movement of the second cursor on the display screen of the first terminal in response to the operation of moving the second cursor.

[0403] Furthermore, the sending unit 2401 is also used to send first shuttle status information to the second terminal, the first shuttle status information being used to indicate the start of shuttle.

[0404] Furthermore, the sending unit 2401 is also used to send data of the operation of moving the first cursor input by the user using the input device of the first terminal to the second terminal, so that the second terminal can display the animation of the first cursor movement on the display screen of the second terminal.

[0405] Furthermore, the drag operation includes a press operation and a move operation; the device may also include an interception unit 2405.

[0406] The interception unit 2405 is used to intercept the press event after the user performs a press operation using the input device of the first terminal.

[0407] The sending unit 2401 is specifically used to send the first operation parameters included in the press event to the second terminal.

[0408] The interception unit 2405 is also configured to intercept a movement event after the user performs a movement operation using the input device of the first terminal.

[0409] The sending unit 2401 is specifically used to send the second operation parameters, including the mobile event, to the second terminal.

[0410] The first and second operation parameters are used to simulate press and move events on the second terminal, respectively, and then to indicate the initiation of dragging the target object.

[0411] Furthermore, the interception unit 2405 is used to intercept the movement event after the user performs a movement operation using the input device of the first terminal.

[0412] The sending unit 2401 is specifically used to send the third operation parameters including the movement event to the second terminal. The third operation parameters are used by the second terminal to simulate the movement event and then to indicate the movement of the first cursor.

[0413] Furthermore, the receiving unit 2402 is also used to receive second shuttle status information from the second terminal, the second shuttle status information being used to indicate the end of the shuttle.

[0414] Furthermore, the sending unit 2401 is also used to send a request message to the second terminal, the request message being used to request drag data.

[0415] Furthermore, the device may also include an input unit 2406.

[0416] Interception unit 2405 is used to cancel the interception of input events.

[0417] The input unit 2406 is used to receive a movement event after the user performs a movement operation using the input device of the first terminal.

[0418] Input unit 2406 is also used to generate press events.

[0419] Display unit 2403 is specifically used to display an animation of an object moving with a second cursor on the display screen of a first terminal based on movement events, press events, and drag data.

[0420] Furthermore, the display unit 2403 is also used to display an invisible window, the transparency of which is greater than a threshold, and the invisible window is used to initiate drag events.

[0421] Furthermore, the long press operation includes both pressing and releasing operations. The interception unit 2405 is used to intercept the press event after the user performs a press operation using the input device of the first terminal.

[0422] The sending unit 2401 is specifically used to send the fourth operation parameters included in the press event to the second terminal.

[0423] The interception unit 2405 is also used to intercept the lift-up event after the user performs a press operation for a period of time and then performs a lift-up operation using the input device of the first terminal.

[0424] The sending unit 2401 is also used to send the fifth operation parameters included in the lift event to the second terminal.

[0425] The fourth and fifth operation parameters are used to simulate press and release events on the second terminal, respectively, and then to trigger the application's interface windowing.

[0426] This application also provides a cross-device object drag-and-drop device, which can be applied to the first terminal or the second terminal in the above embodiments. The device may include: a processor, and a memory for storing processor-executable instructions; wherein the processor is configured to execute instructions to implement the various functions or steps performed by the PC, mobile phone, or tablet computer in the above method embodiments.

[0427] This application also provides a terminal (which may be the first terminal or the second terminal in the above embodiments), which may include: a display screen, a memory, and one or more processors. The display screen, memory, and processor are coupled. The memory is used to store computer program code, which includes computer instructions. When the processor executes the computer instructions, the terminal can perform various functions or steps performed by a PC, mobile phone, or tablet computer in the above method embodiments. Of course, the terminal includes, but is not limited to, the display screen, memory, and one or more processors described above. For example, the structure of the terminal can be referred to... Figure 2AThe structure of the mobile phone is shown.

[0428] This application also provides a chip system that can be applied to the terminal (such as the first terminal or the second terminal) in the foregoing embodiments. Figure 25 As shown, the chip system includes at least one processor 2501 and at least one interface circuit 2502. The processor 2501 can be the processor in the aforementioned terminal. The processor 2501 and the interface circuit 2502 can be interconnected via a line. The processor 2501 can receive and execute computer instructions from the memory of the aforementioned terminal through the interface circuit 2502. When the computer instructions are executed by the processor 2501, the terminal can perform the various steps executed by the PC, mobile phone, or tablet computer in the aforementioned embodiments. Of course, the chip system may also include other discrete components, which are not specifically limited in this application embodiment.

[0429] This application also provides a computer-readable storage medium for storing computer instructions executed by the aforementioned terminal (such as a PC, mobile phone, or tablet computer).

[0430] This application also provides a computer program product, including computer instructions that run on the aforementioned terminal (such as a PC, mobile phone, or tablet computer).

[0431] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0432] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0433] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0434] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0435] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially or in other words, the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0436] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A cross-device object-dragging method, characterized by, Applied to a second terminal connected to a first terminal, wherein the first terminal and the second terminal run different operating systems, the method includes: The second terminal creates a virtual input device; The second terminal displays a first cursor on an object displayed on the second terminal; the object is text or a file; the second terminal receives a first operation parameter from the first terminal, the first operation parameter being the operation parameter contained in the press event received by the first terminal after the user performs a press operation using the input device of the first terminal; The second terminal receives a second operation parameter from the first terminal. The second operation parameter is the operation parameter contained in the movement event received by the first terminal after the user performs a movement operation using the input device of the first terminal. The second terminal uses the virtual input device to simulate the press event according to the first operation parameter and the move event according to the second operation parameter, wherein the press event and the move event are used to indicate the initiation of dragging the object; In response to the press event and the move event, the second terminal displays an animation on its display screen of the object's shadow moving with the first cursor; The second terminal sends drag data to the first terminal; wherein, the drag data includes drag event content and the bitmap of the shadow; when the object is text, the drag event content includes the text, and when the object is a file, the drag event content is the file path.

2. The method according to claim 1, characterized in that, The drag data is used by the first terminal to display an animation of the object moving with the second cursor on the display screen of the first terminal.

3. The method according to claim 1, characterized in that, The method further includes: The second terminal creates a view control with an transparency greater than a threshold, and the view control is used to initiate drag events.

4. The method according to claim 1, characterized in that, After the second terminal displays an animation of the object moving with the first cursor on its display screen in response to the press event and the move event, the method further includes: The second terminal acquires and caches the drag-and-drop data; Sending drag data to the first terminal includes: The second terminal sends the cached drag data to the first terminal.

5. The method according to claim 4, characterized in that, Before sending the drag data to the first terminal, the method further includes: The second terminal receives a request message from the first terminal, the request message being used to request the dragged data.

6. The method according to claim 1, characterized in that, Before the second terminal displays the first cursor on the object displayed on the second terminal, the method further includes: The second terminal receives first shuttle status information from the first terminal, the first shuttle status information being used to indicate the start of shuttle; In response to the first shuttle status information, the second terminal displays the first cursor on the second terminal.

7. The method according to claim 6, characterized in that, After the second terminal displays the first cursor on the second terminal, the method further includes: The second terminal receives the operation of moving the first cursor input by the user using the input device of the first terminal; In response to the operation of moving the first cursor, the second terminal displays an animation of the movement of the first cursor on the display screen of the second terminal until the first cursor moves onto the object.

8. The method according to claim 7, characterized in that, The second terminal receives an operation from the user, inputting a movement of the first cursor using the input device of the first terminal, including: The second terminal receives a third operation parameter from the first terminal. The third operation parameter is the operation parameter contained in the movement event received by the first terminal after the user performs a movement operation using the input device of the first terminal. The second terminal simulates the movement event according to the third operation parameter, and the movement event is used to indicate the movement of the first cursor.

9. The method according to claim 1, characterized in that, After the second terminal sends drag data to the first terminal, the method further includes: The second terminal sends a second shuttle status information to the first terminal, which is used to indicate the end of the shuttle.

10. The method according to any one of claims 1-9, characterized in that, The second terminal creates a virtual input device, including: After the second terminal successfully establishes a connection with the first terminal, it creates a virtual input device; or, The second terminal receives a notification message from the first terminal, and in response to the notification message, the second terminal creates the virtual input device.

11. A cross-device object dragging method, characterized in that, Applied to a first terminal connected to a second terminal, wherein the first terminal and the second terminal run different operating systems, the method includes: After the user performs a press operation using the input device of the first terminal, the first terminal intercepts the press event corresponding to the press operation; The first terminal sends the first operation parameters included in the press event to the second terminal; After the user performs a movement operation using the input device of the first terminal, the first terminal intercepts the movement event corresponding to the movement operation. The first terminal sends the second operation parameters, including the movement event, to the second terminal; The first operation parameter and the second operation parameter are respectively used by the second terminal to simulate the press event and the move event using a virtual input device, and then used to instruct the initiation of dragging the object displayed on the second terminal. The virtual input device is created by the second terminal. The object is text or a file. The first terminal receives drag data from the second terminal, the drag data being sent from the second terminal to the first terminal; wherein, the drag data includes drag event content and a bitmap of shadow; when the object is text, the drag event content includes the text, and when the object is a file, the drag event content is the file path; The first terminal displays an animation of the object's shadow moving with the second cursor on its display screen based on the drag data, the press event, and the move event.

12. The method according to claim 11, characterized in that, Before the first terminal sends the first operation parameter included in the press event and the second operation parameter included in the movement event to the second terminal, the method further includes: The first terminal determines that the second cursor has slid off the edge of the first terminal's display screen.

13. The method according to claim 12, characterized in that, Before the first terminal determines that the second cursor has slid off the edge of the first terminal's display screen, the method further includes: The first terminal receives an operation from the user, using the input device of the first terminal, to move the second cursor; In response to the operation of moving the second cursor, the first terminal displays an animation of the movement of the second cursor on the display screen of the first terminal.

14. The method according to claim 12, characterized in that, After the first terminal determines that the second cursor has slid off the edge of the first terminal's display screen, the method further includes: The first terminal sends first shuttle status information to the second terminal, which is used to indicate that the shuttle has started.

15. The method according to claim 12, characterized in that, After the first terminal determines that the second cursor has slid off the edge of the first terminal's display screen, the method further includes: The first terminal sends data to the second terminal, which is input by the user using the input device of the first terminal to move the first cursor, so that the second terminal can display the animation of the first cursor movement on the display screen of the second terminal.

16. The method according to claim 15, characterized in that, The first terminal sends data to the second terminal, including data input by the user using the input device of the first terminal to move the first cursor. After the user performs a movement operation using the input device of the first terminal, the first terminal intercepts the movement event; The first terminal sends a third operation parameter, which includes the movement event, to the second terminal. The third operation parameter is used by the second terminal to simulate the movement event and then to instruct the movement of the first cursor.

17. The method according to claim 11, characterized in that, Before the first terminal displays an animation of the object's shadow moving with the second cursor on the first terminal's display screen based on the drag data, the press event, and the move event, the method further includes: The first terminal receives second shuttle status information from the second terminal, which is used to indicate the end of the shuttle.

18. The method according to claim 17, characterized in that, Before the first terminal receives drag data from the second terminal, the method further includes: The first terminal sends a request message to the second terminal, the request message being used to request the dragged data.

19. The method according to claim 17, characterized in that, After the first terminal receives the second shuttle status information from the second terminal, the method further includes: The first terminal cancels the interception of input events; The first terminal displays an animation of the object moving with the second cursor on its screen based on the drag data, the press event, and the move event, including: After the user performs a movement operation using the input device of the first terminal, the first terminal receives the movement event; The first terminal generates a press event; The first terminal displays an animation of the object moving with the second cursor on its screen based on the movement event, the press event, and the drag data.

20. The method according to claim 12, characterized in that, The method further includes: The first terminal displays an invisible window with an transparency greater than a threshold, and the invisible window is used to initiate drag events.

21. A cross-device object dragging device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to implement the method as described in any one of claims 1-10, or the method as described in any one of claims 11-20, when executing the instructions.

22. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method as described in any one of claims 1-10, or the method as described in any one of claims 11-20.

23. A cross-device object drag-and-drop system, characterized in that, include: A first terminal and a second terminal; the first terminal is connected to the second terminal, and the first terminal and the second terminal run different operating systems; The second terminal is used to create a virtual input device; Display a first cursor on the object shown on the second terminal; The object is text or a file; receiving a first operation parameter from the first terminal, the first operation parameter being the operation parameter contained in a press event received by the first terminal after the user performs a press operation using the input device of the first terminal; receiving a second operation parameter from the first terminal, the second operation parameter being the operation parameter contained in a movement event received by the first terminal after the user performs a movement operation using the input device of the first terminal; The virtual input device is used to simulate the press event according to the first operation parameter and the move event according to the second operation parameter. The press event and the move event are used to indicate the initiation of dragging the object. In response to the press event and the move event, an animation of the shadow of the object moving with the first cursor is displayed on the screen of the second terminal; drag data is sent to the first terminal; wherein the drag data includes drag event content and a bitmap of the shadow; when the object is text, the drag event content includes the text, and when the object is a file, the drag event content is a file path; The first terminal is configured to intercept the press event and the move event; receive drag data from the second terminal; and display an animation of the shadow of the object moving with the second cursor on the display screen of the first terminal according to the drag data, the press event and the move event.

24. The system according to claim 23, characterized in that, The second terminal is also used to acquire and cache the dragged data; Sending drag data to the first terminal includes: sending the cached drag data to the first terminal.

25. The system according to claim 23, characterized in that, The first terminal is also used to determine when the second cursor slides off the edge of the display screen of the first terminal.

26. The system according to claim 23, characterized in that, The first terminal is further configured to send data of a drag operation input by the user using the input device of the first terminal to the second terminal after determining that the second cursor has slid out of the edge of the display screen of the first terminal.

27. The system according to claim 23, characterized in that, The first terminal is further configured to send first shuttle status information to the second terminal after determining that the second cursor has slid out of the edge of the display screen of the first terminal, the first shuttle status information being used to indicate that the shuttle has started.

28. The system according to claim 27, characterized in that, The second terminal is further configured to display the first cursor on the second terminal; receive an operation input by a user using an input device of the first terminal to move the first cursor; and, in response to the operation of moving the first cursor, display an animation of the movement of the first cursor on the display screen of the second terminal until the first cursor moves onto the object.

29. The system according to claim 28, characterized in that, The receiving of the user's input of the operation to move the first cursor using the input device of the first terminal includes: Receive a third operation parameter from the first terminal, wherein the third operation parameter is the operation parameter contained in the movement event received by the first terminal after the user performs a movement operation using the input device of the first terminal; The movement event is simulated according to the third operation parameter, and the movement event is used to indicate the movement of the first cursor.

30. The system according to claim 23, characterized in that, The second terminal is further configured to send second shuttle status information to the first terminal after determining that the object has been dragged out of the edge of the display screen of the second terminal, the second shuttle status information being used to indicate the end of the shuttle.

31. The system according to any one of claims 23-30, characterized in that, The creation of the virtual input device includes: After a successful connection is established with the first terminal, a virtual input device is created; or, Upon receiving a notification message from the first terminal, the virtual input device is created in response to the notification message.

Images