Cross-device dragging method for window, device, and storage medium

Abstract

The present application provides a cross-device dragging method for a window, a device, and a storage medium. The method is applied to a first electronic device having a Windows system and a second electronic device having an Android system. When a mouse used for dragging a first window displayed in the first electronic device to move traverses to the second electronic device, the first electronic device transmits a window traversal indication to the second electronic device, so that the second electronic device can first start a first floating window that does not display the first window, such that a user can visually feel that the first window is subjected to traversal. When the mouse is released, the first electronic device notifies the second electronic device that the user stops pressing the left key of the mouse, so that the second electronic device can release the first floating window and start a second floating window as an extended screen of the first electronic device. Therefore, the first window can be displayed in a display area of the second floating window, thereby achieving cross-device dragging of a window in a keyboard and mouse sharing scenario.

Description

Cross-device dragging method, device, and storage medium for windows

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on July 11, 2023, with application number 202310849520.9, and invention name “Cross-device dragging method, device and storage medium for windows”, and the Chinese patent application filed with the China Patent Office on July 11, 2023, with application number 202310850928.8, and invention name “Cross-device dragging method, device and storage medium for windows”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of electronic devices, and in particular to a method, device, and storage medium for dragging a window across devices. Background Art

[0003] The so-called keyboard and mouse sharing means that multiple electronic devices that have established a keyboard and mouse sharing connection can share a set of input / output (I / O) devices such as a keyboard and mouse. That is, in a keyboard and mouse sharing scenario, a user can use the same mouse to operate multiple electronic devices.

[0004] However, with the diversification of user needs, simply using the same mouse to traverse different electronic devices and then operate different electronic devices can no longer meet the user's needs.

[0005] Summary of the Invention

[0006] In order to solve the above technical problems, the present application provides a method, device and storage medium for dragging windows across devices, aiming to realize cross-device dragging of windows in a keyboard and mouse sharing scenario, especially dragging windows from Windows devices to Android devices, so as to better meet user usage needs and enhance user experience.

[0007] In a first aspect, the present application provides a method for dragging a window across devices, which is applied to a system including a first electronic device and a second electronic device, wherein the operating system of the first electronic device is a Windows system and the operating system of the second electronic device is an Android system. The method includes: a first electronic device displays a first window; the first electronic device responds to a user operating a mouse cursor to drag the first window and drags the first window to move within the screen of the first electronic device; when the mouse cursor moves out of the screen edge of the first electronic device, the first electronic device sends a window crossing indication for the first window to the second electronic device, and hides the first window displayed on the first electronic device; wherein, the mouse corresponding to the mouse cursor is connected to the first electronic device or the second electronic device; the second electronic device responds to the window crossing indication and displays a first floating window on the screen of the second electronic device, and the first window is not displayed in the first floating window; in the process of the mouse cursor dragging the first floating window to move within the screen of the second electronic device, the first electronic device responds to the user releasing the left mouse button and sends a first indication to the second electronic device; the second electronic device responds to the first indication and releases the first floating window, and displays a second floating window on the screen of the second electronic device, and the second floating window covers the first floating window; when the second electronic device receives the first collaborative screen sent by the first electronic device, it displays the first collaborative screen in the display area of ​​the second floating window, and the first collaborative screen includes all the display content of the first window.

[0008] The first electronic device is, for example, a PC with a Windows system as described in the following embodiment, and the second electronic device is, for example, a tablet with an Android system as described in the following embodiment.

[0009] The first window is, for example, the window 20a mentioned in the following embodiment.

[0010] The first floating window is, for example, the floating window 40a mentioned in the following embodiment, and the second floating window is, for example, the floating window 50a mentioned in the following embodiment.

[0011] The first collaborative screen is, for example, the window 20b and the task bar 30b displayed in the display area of ​​the floating window 50a as described in the following embodiment.

[0012] In the keyboard and mouse sharing scenario, various instructions sent by the first electronic device to the second electronic device, and various messages sent by the second electronic device to the first electronic device, are sent based on the keyboard and mouse sharing connection established between the first electronic device and the second electronic device.

[0013] The first collaborative screen, and the second collaborative screen described below, are transmitted based on the extended connection (screen collaborative connection) established between the first electronic device and the second electronic device.

[0014] In this aspect, the first electronic device and the second electronic device establish a keyboard and mouse sharing connection, that is, they are in a keyboard and mouse sharing scenario.

[0015] Regarding the specific implementation process of dragging the first window from the first electronic device to the second electronic device, please refer to the description of steps 101 to 139 and steps 201 to 227 in the following embodiments, which will not be repeated here.

[0016] This allows you to drag windows across devices in a keyboard and mouse sharing scenario, especially by dragging a window from a Windows device to an Android device.

[0017] According to the first aspect, after displaying the first collaborative screen in the display area of ​​the second floating window, the method also includes: the first electronic device displays the second window; the first electronic device drags the second window to move within the screen of the first electronic device in response to the user operating the mouse cursor to drag the second window; when the mouse cursor moves out of the screen edge of the first electronic device, the first electronic device sends a second collaborative screen for the second window to the second electronic device, the second collaborative screen includes a first part of the second window, and displays the second part of the second window on the screen of the first electronic device; wherein the first part is the part that moves out of the screen edge of the first electronic device, and the second part is the part that does not move out of the screen edge of the first electronic device; when the second electronic device receives the second collaborative screen sent by the first electronic device, it displays the second collaborative screen in the display area of ​​the second floating window; wherein the window state of the second window in the second collaborative screen is a restored state, and the second collaborative screen blocks the first collaborative screen; wherein, when the window state is a restored state, the title bar of the first window includes a maximize control but does not include a restore control.

[0018] The second window displayed on the first electronic device is, for example, window 70a corresponding to document 1 shown in (1) of Figure 18. The second collaborative screen displayed in the second floating window is, for example, window 70b shown in (3) and (4) of Figure 18.

[0019] The generation process of the second collaborative screen also requires the use of the synthetic drawing module. Since the first collaborative screen including the task bar of the first electronic device has been displayed in the second floating window, when generating the second collaborative screen, it is only necessary to generate it based on the second window.

[0020] Regarding the process of dragging the second window in the first electronic device to the second floating window already displayed in the second electronic device, please refer to the embodiment section of Figure 18 below, which will not be repeated here.

[0021] According to the first aspect, or any implementation of the first aspect above, when the mouse cursor moves out of the screen edge of the first electronic device, the first electronic device sends a window crossing indication for the first window to the second electronic device, and hides the first window displayed on the first electronic device, including: when the mouse cursor moves out of the screen edge of the first electronic device, the first electronic device sends a window crossing indication for the first window to the second electronic device; after the first electronic device sends the window crossing indication to the second electronic device, when it receives a window crossing confirmation message made by the second electronic device for the window crossing indication, it hides the first window displayed on the first electronic device.

[0022] Regarding the mouse cursor, the crossing of the first window, and the hiding of the first window, please refer to Figures 12, 13, 14A, 14B, and steps 110 to 113, and step 207, and steps 114 to 117 in the following embodiments, which will not be repeated here.

[0023] According to the first aspect, or any implementation of the first aspect above, after the second electronic device displays the first floating window on the screen of the second electronic device in response to the window crossing indication, the method also includes: the second electronic device generates a window crossing confirmation message for the window crossing indication and sends the window crossing message to the first electronic device.

[0024] Regarding the operations before and after the second electronic device sends the window crossing confirmation message, and the specific process of sending the window crossing message to the first electronic device, please refer to steps 113 and 201 to 207 in the following embodiments, which will not be repeated here.

[0025] According to the first aspect, or any implementation method of the first aspect above, the first indication includes a release window indication and a screen collaboration indication; the release window indication is used to instruct the second electronic device to release the first floating window; the screen collaboration indication is used to instruct the second electronic device to establish a screen collaboration connection with the first electronic device in which the collaboration mode is an extended mode; wherein, the second floating window is displayed on the screen of the second electronic device after the screen collaboration connection is successfully established.

[0026] The screen collaboration indication mentioned in this aspect is, for example, the extended mode indication mentioned in the following embodiments.

[0027] Regarding the operations performed by the second electronic device according to the release window indication and the screen coordination indication, please refer to steps 129, 217 to 221, 134, and 222 to 227 in the following embodiments, which will not be repeated here.

[0028] According to the first aspect, or any implementation of the first aspect above, the second electronic device releases the first floating window in response to the first indication, and after the second floating window is displayed on the screen of the second electronic device, the method also includes: the second electronic device generates a screen collaborative connection success message for the screen collaborative indication, and sends the screen collaborative connection success message to the first electronic device.

[0029] The screen collaboration indication in this aspect is, for example, the extended mode indication in the following embodiment. The screen collaboration connection success message is, for example, the extended connection success message in the following embodiment.

[0030] For the specific implementation details of the extended connection success message (screen collaborative connection success message) made by the second electronic device in response to the extended mode indication (screen collaborative indication) after releasing the first floating window, please refer to step 134 and steps 222 to 227 in the following embodiment, which will not be repeated here.

[0031] According to the first aspect, or any implementation of the first aspect above, the first collaborative screen also includes a taskbar of the first electronic device; the method also includes: the first electronic device sets the window state of the first window to a maximized state in response to a screen collaborative connection success message, so that the first window displays all the display content included; the first electronic device synthesizes and draws the first collaborative screen based on the first window with the window state in the maximized state and the taskbar of the first electronic device; and sends the obtained first collaborative screen to the second electronic device.

[0032] The task bar of the first electronic device is, for example, the task bar 30a described in the following embodiment and drawings. Specific implementation details of synthesizing and drawing the first collaborative screen can be found in step 138 of the following embodiment, which will not be repeated here.

[0033] According to the first aspect, or any implementation of the first aspect above, when the window state is maximized, the title bar of the first window includes a restore control but does not include a maximize control.

[0034] Regarding the style of the title bar of the first window in the maximized state, for example, the title bar of the window (window 20b) corresponding to document 2 displayed on the Android device in FIG3E .

[0035] According to the first aspect, or any implementation of the first aspect above, a first electronic device hides a first window displayed on the first electronic device, including: the first electronic device determines, based on a handle of the first window, a width and height of the first window; the first electronic device determines, based on mouse movement information, a current X-coordinate and Y-coordinate of a mouse cursor corresponding to the mouse; the first electronic device determines, based on the current X-coordinate and Y-coordinate of the mouse cursor, the width of the first window, and an X-coordinate interval corresponding to the screen of the first electronic device, hidden coordinates for hiding the first window; wherein the X-coordinate interval includes an X-axis coordinate of a left edge of the screen of the first electronic device and an X-axis coordinate of a right edge of the screen of the first electronic device, the X-axis coordinate of the left edge of the screen being smaller than the X-axis coordinate of the right edge of the screen; and the first electronic device hides the first window at the location of the hidden coordinates.

[0036] Regarding the method for determining the hidden coordinates, please refer to the description of the embodiment shown in FIG12 , which will not be repeated here.

[0037] According to the first aspect, or any implementation of the first aspect above, the first electronic device determines hidden coordinates for hiding the first window based on the current X-coordinate and Y-coordinate of the mouse cursor, the width of the first window, and the X-coordinate interval corresponding to the screen of the first electronic device, including: when the X-coordinate of the mouse cursor is not greater than the X-axis coordinate of the left edge of the screen, the first electronic device determines the difference between the X-axis coordinate of the left edge of the screen and the width of the first window; and the first electronic device uses the difference as the X-coordinate of the hidden coordinate and uses the Y-coordinate of the mouse cursor as the Y-coordinate of the hidden coordinate.

[0038] Regarding the hidden coordinates corresponding to when the first window crosses the left edge of the screen of the Windows device, please refer to the description of the embodiment shown in Figure 14B, which will not be repeated here.

[0039] According to the first aspect, or any implementation of the first aspect above, the first electronic device determines the hidden coordinates for hiding the first window based on the current X coordinate and Y coordinate of the mouse cursor, the width of the first window, and the X coordinate interval corresponding to the screen of the first electronic device, including: when the X coordinate of the mouse cursor is not less than the X-axis coordinate of the right edge of the screen, the first electronic device uses the X-axis coordinate of the right edge of the screen as the X coordinate of the hidden coordinate, and uses the Y coordinate of the mouse cursor as the Y coordinate of the hidden coordinate.

[0040] Regarding the hidden coordinates corresponding to when the first window crosses the right edge of the screen of the Windows device, please refer to the description of the embodiment shown in Figure 14A, which will not be repeated here.

[0041] According to the first aspect, or any implementation of the first aspect above, after the first electronic device hides the first window at the location of the hidden coordinates, the method also includes: in response to a drag operation of the first window from the second electronic device back to the first electronic device, displaying the first window from the hidden coordinates.

[0042] According to the first aspect, or any implementation of the first aspect above, the first electronic device displays a first window, including: the first electronic device displays the first window in a maximized state; wherein, when the window state is maximized, the title bar of the first window includes a restore control but does not include a maximize control.

[0043] That is, before the first window is dragged, the first window occupies the entire display area of ​​the Windows device screen. In this scenario, the style of the title bar of the first window is the title bar of the window (window 20b) corresponding to document 2 displayed on the Android device in Figure 3E.

[0044] According to the first aspect, or any implementation method of the first aspect above, the first electronic device drags the first window to move within the screen of the first electronic device in response to the user operating the mouse cursor to drag the first window, including: the first electronic device switches the window state of the first window from a maximized state to a restored state in response to the user operating the mouse cursor to drag the first window; wherein, when the window state is the restored state, the title bar of the first window includes a maximize control but does not include a restore control; the first electronic device drags the first window whose window state is the restored state to move within the screen of the first electronic device.

[0045] That is, for the first window displayed in the maximized state, it is automatically switched to the restored state first, and then dragged and moved.

[0046] According to the first aspect, or any implementation of the first aspect above, a first electronic device displays a first window, including: the first electronic device displays the first window in a restored state; wherein, when the window is in the restored state, a title bar of the first window includes a maximize control but does not include a restore control. That is, the title bar of the first window is styled like the title bar of the window (window 20a) corresponding to document 2 displayed on the Windows device in FIG2A.

[0047] According to the first aspect, or any implementation of the first aspect above, the first electronic device, in response to a user operating a mouse cursor to drag the first window, drags the first window within the screen of the first electronic device, including: the first electronic device, in response to the user operating the mouse cursor to drag the first window, drags the first window, which is in a restored state, within the screen of the first electronic device. That is, directly dragging the window (window 20a) corresponding to document 2 displayed on the Windows device in FIG. 2A .

[0048] According to the first aspect, or any implementation of the first aspect above, during the process of dragging the first window to move within the screen of the first electronic device, the method also includes: during the process of dragging the first window to move within the screen of the first electronic device, the first electronic device determines, based on the movement information of the mouse, whether the mouse cursor corresponding to the mouse moves out of the screen of the first electronic device from the left edge of the screen of the first electronic device, or the right edge of the screen; when it is determined that the mouse cursor corresponding to the mouse moves out of the screen of the first electronic device from the left edge of the screen of the first electronic device, or the right edge of the screen, the first electronic device determines that the mouse cursor moves out of the screen edge of the first electronic device.

[0049] Regarding the implementation of this aspect, please refer to the specific description of step 109 and step 110 in the following embodiment, which will not be repeated here.

[0050] According to the first aspect, or any implementation method of the first aspect above, the first electronic device drags the first window to move within the screen of the first electronic device in response to the user operating the mouse cursor to drag the first window, including: when the first electronic device monitors the window move message of the first window and the left mouse button press message of the left mouse button being pressed, it determines that the user operates the mouse cursor to perform a drag operation on the first window; the first electronic device responds to the drag operation and drags the first window to move within the screen of the first electronic device.

[0051] Regarding the user operating the mouse cursor to drag the first window, and the specific operations performed by the first electronic device in response to the dragging operation, please refer to the description of steps 101 to 107 in the following embodiment, which will not be repeated here.

[0052] According to the first aspect, or any implementation of the first aspect above, the first electronic device sends a first indication to the second electronic device in response to the user releasing the left mouse button, including: when the first electronic device monitors the release message of the left mouse button and monitors the message that the first window stops moving, it determines that the user has released the left mouse button; the first electronic device sends the first indication to the second electronic device in response to the release operation.

[0053] The first indication may include the release window indication in step 125 and the extended mode indication (screen coordination indication) in step 131 in the following embodiment.

[0054] Regarding the specific implementation details of the user releasing the left mouse button and the first electronic device sending the first indication to the second electronic device in response to the release operation, please refer to the description of steps 123 to 134 in the following embodiment, which will not be repeated here.

[0055] According to the first aspect, or any implementation of the first aspect above, the second electronic device includes a screen computer and a foldable screen mobile phone.

[0056] Therefore, by selecting an electronic device with a relatively large screen as the second electronic device, the second floating window displayed on the second electronic device can be relatively large, and the first window dragged from the first electronic device to the second floating window of the second electronic device can be tiled, making it easier for the user to view the content in the window.

[0057] In a second aspect, the present application provides a method for dragging a window across devices, which is applied to a system including a first electronic device and a second electronic device, wherein the operating system of the first electronic device is a Windows system and the operating system of the second electronic device is an Android system. The method includes: the first electronic device displays a first window; the first electronic device responds to the user's operation of dragging the first window by operating the mouse cursor to drag the first window, and drags the first window to move within the screen of the first electronic device; when the mouse cursor moves out of the screen edge of the first electronic device, the first electronic device sends a window crossing indication for the first window to the second electronic device, and hides the first window displayed on the first electronic device; wherein the mouse corresponding to the mouse cursor is connected to the first electronic device or the second electronic device; the second electronic device responds to the window crossing indication and displays a first floating window on the screen of the second electronic device, the first floating window including a first upper status bar, a first lower status bar and a first display area, and the first window is not displayed in the first display area; the second electronic device A mouse cursor is displayed in the first upper status bar, and a left mouse button press event is registered for the mouse; while the mouse cursor drags the first floating window to move within the screen of the second electronic device, the first electronic device sends a first indication to the second electronic device in response to the user releasing the left mouse button; the second electronic device registers a left mouse button lift event for the mouse in response to the first indication to release the first floating window and display a second floating window, which covers the first floating window and includes a second upper status bar, a second lower status bar, and a second display area; when the second electronic device receives the first collaborative screen sent by the first electronic device, it displays the first collaborative screen in the second display area, and the first collaborative screen includes all display contents of the first window.

[0058] The first electronic device is, for example, a PC with a Windows system as described in the following embodiment, and the second electronic device is, for example, a tablet with an Android system as described in the following embodiment.

[0059] The first window is, for example, the window 20a mentioned in the following embodiment.

[0060] The first floating window is, for example, the floating window 40a mentioned in the following embodiment, and the second floating window is, for example, the floating window 50a mentioned in the following embodiment.

[0061] The first collaborative screen is, for example, the window 20b and the task bar 30b displayed in the display area of ​​the floating window 50a as described in the following embodiment.

[0062] In the keyboard and mouse sharing scenario, various instructions sent by the first electronic device to the second electronic device, and various messages sent by the second electronic device to the first electronic device, are sent based on the keyboard and mouse sharing connection established between the first electronic device and the second electronic device.

[0063] The first collaborative screen, and the second collaborative screen described below, are transmitted based on the extended connection (screen collaborative connection) established between the first electronic device and the second electronic device.

[0064] In this aspect, the first electronic device and the second electronic device establish a keyboard and mouse sharing connection, that is, they are in a keyboard and mouse sharing scenario.

[0065] Among them, the first upper status bar is, for example, the status bar 1 of the floating window 40a mentioned in the following embodiment; the first lower status bar is, for example, the status bar 2 of the floating window 40a mentioned in the following embodiment; the first display area, that is, the area after the first upper status bar and the first lower status bar are removed from the first floating window, is, for example, the area where the control 40a-5 is displayed in the floating window 40a mentioned in the following embodiment.

[0066] Among them, the second upper status bar is, for example, the status bar 1 of the floating window 50a mentioned in the following embodiment; the second lower status bar is, for example, the status bar 2 of the floating window 50a mentioned in the following embodiment; the second display area, that is, the area after the second floating window removes the first upper status bar and the first lower status bar, is, for example, the area where the window 20b and the taskbar 30b are displayed in the floating window 50a mentioned in the following embodiment.

[0067] Among them, it is the left mouse button press event registered by the mouse, that is, the down event mentioned in the following embodiment.

[0068] Among them, it is the left mouse button lift event registered by the mouse, that is, the up event mentioned in the following embodiment.

[0069] Regarding the specific implementation process of dragging the first window from the first electronic device to the second electronic device, please refer to the description of steps 101 to 139 and steps 201 to 227 in the following embodiments, which will not be repeated here.

[0070] This allows you to drag windows across devices in a keyboard and mouse sharing scenario, especially by dragging a window from a Windows device to an Android device.

[0071] According to the second aspect, when the mouse cursor moves out of the screen edge of the first electronic device, the first electronic device sends a window crossing indication for the first window to the second electronic device, and hides the first window displayed on the first electronic device. The method also includes: the second electronic device establishes a keyboard and mouse sharing connection with the first electronic device, and in the process of establishing the keyboard and mouse sharing connection, agrees on the aspect ratio of the first display area; wherein the aspect ratio of the first display area is the same as the aspect ratio of the screen of the first electronic device.

[0072] Therefore, in the process of establishing a keyboard and mouse sharing connection between the first electronic device and the second electronic device, the aspect ratio followed by the two in the screen collaboration scenario is predetermined in advance. In this way, in the screen collaboration scenario, the first electronic device can generate a first collaborative screen according to the agreed aspect ratio, and the area used to display the collaborative screen in the second electronic device meets the aspect ratio. Therefore, there will be no black edges on the collaborative screen subsequently displayed in the second display area, which makes the user feel that the first window is directly dragged to the second electronic device for display, thereby improving the user experience.

[0073] According to the second aspect, or any implementation method of the above second aspect, the heights of the first upper status bar and the first lower task bar are fixed; the second electronic device displays the first floating window on the screen of the second electronic device in response to the window crossing indication, including: the second electronic device creates a first Activity in response to the window crossing indication according to the aspect ratio, the screen size of the second electronic device, the height of the first upper status bar, the height of the first lower task bar and the preset first window ratio; and, the window mode of the first Activity is set to the floating window mode to obtain the first floating window.

[0074] Understandably, in the Android system, each window is composed of an Activity component. Therefore, after drawing the first Activity, setting its window mode to the floating window mode can obtain the first floating window.

[0075] According to the second aspect, or any implementation method of the above second aspect, the second electronic device creates a first Activity in response to a window crossing indication according to the aspect ratio, the screen size of the second electronic device, the height of the first upper status bar, the height of the first lower task bar and the preset first window ratio, including: the second electronic device determines the width of the first display area and the height of the first display area according to the aspect ratio, the screen size of the second electronic device and the preset first window ratio; the second electronic device obtains the Y coordinate of the first window crossing carried in the window crossing indication, and uses the Y coordinate of the first window crossing as the starting Y coordinate of the height of the first Activity; the second electronic device determines the ending Y coordinate of the height of the first Activity based on the starting Y coordinate, the height of the first upper status bar, the height of the first display area and the height of the first lower task bar; the second electronic device sets the first bounds attribute of the first Activity based on the width, starting Y coordinate and ending Y coordinate of the first display area; the second electronic device creates the first Activity based on the first bounds attribute.

[0076] The starting Y coordinate of the height of the first Activity is, for example, the top coordinate in the following embodiment, and the ending Y coordinate of the height of the first Activity is, for example, the bottom coordinate in the following embodiment.

[0077] Regarding the setting of the first bounds attribute of the first Activity, please refer to the operation performed by the window management module in the embodiment shown in Figure 16A below. The specific implementation details are not repeated here.

[0078] According to the second aspect, or any implementation of the second aspect above, displaying the second floating window includes: the second electronic device creates a second Activity based on the first Activity; and setting the window mode of the second Activity to a floating window mode to obtain the second floating window.

[0079] Regarding the specific implementation details of creating a second Activity based on the first Activity, that is, creating a second floating window based on the first floating window, please refer to the operations performed by HwActivityStartEx in the embodiment shown in Figure 16A below. The specific implementation details will not be repeated here.

[0080] According to the second aspect, or any implementation of the second aspect above, after displaying the second floating window, the method further includes: the second electronic device destroying the first floating window.

[0081] For example, the destruction operation of the first floating window may be, for example, calling finish(). For the calling logic of this interface, please refer to the operation of HwActivityStartEx in the embodiment shown in FIG16A below. The specific implementation details will not be repeated here.

[0082] This reduces the resource usage of the second electronic device.

[0083] According to the second aspect, or any implementation method of the above second aspect, the second upper status bar is the same height as the first upper status bar, and the second lower status bar is the same height as the first lower task bar; the second electronic device creates a second Activity based on the first Activity, including: the second electronic device obtains the first bounds attribute of the first Activity; when obtaining the first bounds attribute, the second electronic device sets the second bounds attribute of the second Activity based on the first bounds attribute; the second electronic device creates a second Activity based on the second bounds attribute.

[0084] Regarding the specific implementation details of creating a second Activity based on the first Activity, that is, creating a second floating window based on the first floating window, please refer to the operations performed by HwActivityStartEx in the embodiment shown in Figure 16A below. The specific implementation details will not be repeated here.

[0085] According to the second aspect, or any implementation of the second aspect, the second bounds attribute is the same as the first bounds attribute.

[0086] In this way, the size and layout of the second floating window can be guaranteed to be exactly the same as those of the first floating window, so that the first floating window can be completely covered, and the process of replacing the first floating window with the second floating window will be unaware to the user, further improving the user experience.

[0087] According to the second aspect, or any implementation of the second aspect above, the method also includes: when the first bounds attribute is not obtained, the second electronic device determines the width of the second display area and the height of the second display area based on the aspect ratio, the screen size of the second electronic device and the preset first window ratio; the second electronic device obtains the Y coordinate of the first window crossing carried in the window crossing indication, and uses the Y coordinate of the first window crossing as the starting Y coordinate of the height of the second Activity; the second electronic device determines the ending Y coordinate of the height of the second Activity based on the starting Y coordinate, the height of the second upper status bar, the height of the second display area and the height of the second lower status bar; the second electronic device sets the second bounds attribute of the second Activity based on the width, starting Y coordinate and ending Y coordinate of the second display area.

[0088] For specific implementation details of creating the second floating window, please refer to the operations performed by HwActivityStartEx in the embodiment shown in Figure 16A below. The specific implementation details will not be repeated here.

[0089] According to the second aspect, or any implementation of the second aspect above, creating the first Activity further includes: setting the window attribute of the first Activity to a disabled scaling mode.

[0090] The setting of the above-mentioned window attributes is achieved, for example, through DisplayContent described in the following embodiment, and the specific implementation details are not repeated here.

[0091] Therefore, by setting the first activity's window properties to disable scaling, the user cannot drag the first floating window when setting the second bounds based on the first bounds. That is, the first bounds remain unchanged, ensuring that the drawn second floating window completely covers the first floating window.

[0092] According to the second aspect, or any implementation method of the above second aspect, it is characterized in that the second electronic device displays the mouse cursor in the first upper status bar, including: the second electronic device selects a coordinate point from the currently displayed first upper status bar as the initial mouse coordinates according to the display position of the first floating window; wherein the initial mouse coordinates avoid the area where the function buttons included in the first upper status bar are located and the hot zone of the side edge of the screen of the second electronic device; the second electronic device displays the mouse cursor at the initial mouse coordinates.

[0093] For implementation details of setting the initial coordinates of the mouse, please refer to the operations performed by the window setting module in the embodiment shown in Figure 16A below. The specific implementation details will not be repeated here.

[0094] According to the second aspect, or any implementation method of the above second aspect, the first indication includes a release window indication and a screen collaboration indication; the release window indication is used to instruct the second electronic device to release the first floating window; the screen collaboration indication is used to instruct the second electronic device to establish a screen collaboration connection with the first electronic device in which the collaboration mode is an extended mode; wherein, the second floating window is displayed on the screen of the second electronic device after the screen collaboration connection is successfully established.

[0095] The screen collaboration indication mentioned in this aspect is, for example, the extended mode indication mentioned in the following embodiments.

[0096] Regarding the operations performed by the second electronic device according to the release window indication and the screen coordination indication, please refer to steps 129, 217 to 221, 134, and 222 to 227 in the following embodiments, which will not be repeated here.

[0097] According to the second aspect, or any implementation of the second aspect above, the method further includes: the second electronic device registers a drag listener for the first upper status bar; wherein the second electronic device is further configured to cancel the drag listener in response to a window release indication.

[0098] Regarding registering a drag listener for the first upper status bar, this is achieved, for example, by using the HwFreeFormCaptionView described in the following embodiment. The specific implementation details will not be repeated here.

[0099] The operation of canceling the drag monitoring is implemented, for example, by the HnPcMouseManager described in the following embodiment, and the specific implementation details are not repeated here.

[0100] According to the second aspect, or any implementation of the above second aspect, before canceling the drag listening, the method also includes: when the drag listening is triggered and the window movement position information sent by the first electronic device is received, the second electronic device moves the display position of the first floating window according to the window movement position information.

[0101] Regarding the processing operations involved in moving the first floating window, please refer to step 122, step 212 to step 216 in the following embodiment, which will not be repeated here.

[0102] According to the second aspect, or any implementation method of the above second aspect, a second floating window is displayed in a second electronic device whose screen status is horizontal; after the first collaborative screen is displayed in the second display area, the method also includes: the second electronic device responds to the screen rotation operation, and displays a third floating window at a preset position from the upper edge of the screen in the vertical state, and the third floating window includes a third upper status bar, a third lower status bar and a third display area; the second electronic device displays the first collaborative screen in the third display area; and destroys or hides the second Activity; wherein the heights of the third upper status bar and the third lower status bar are fixed, the aspect ratio of the third display area is the same as the aspect ratio of the first display area agreed with the first electronic device when establishing a keyboard and mouse sharing connection, and the aspect ratio of the first display area is the same as the aspect ratio of the screen of the first electronic device.

[0103] For example, when the second floating window is the floating window 50a in the following embodiment, the third floating window can be understood as the floating window 50a in the vertical screen state, such as the state shown in (2) in Figure 7.

[0104] Correspondingly, the third upper status bar, the third lower status bar and the third display area are status bar 1, status bar 2 and the area for displaying window 20b and taskbar 30b in the floating window 50a in the vertical state.

[0105] Regarding the switching between the horizontal and vertical screens, and the position and size changes of the floating window 50a during the switching process, please refer to the description of the embodiment shown in FIG. 7 , which will not be repeated here.

[0106] According to the second aspect, or any implementation of the second aspect above, the method also includes: the second electronic device scales the second floating window in response to the scaling operation on the second floating window; wherein the aspect ratio of the second display area of ​​the enlarged or reduced second floating window is the same as the aspect ratio of the first display area agreed with the first electronic device when establishing a keyboard and mouse sharing connection, and the aspect ratio of the first display area is the same as the aspect ratio of the screen of the first electronic device.

[0107] Therefore, by setting the second floating window during the scaling process, the second display area also meets the agreed aspect ratio, so that no matter how the second floating window is increased, no black border will appear during the display of the first collaborative screen.

[0108] Regarding the zooming operation on the second floating window, please refer to the description of the embodiments shown in Figures 6A and 6B below, which will not be repeated here.

[0109] In a third aspect, the present application provides an electronic device. The electronic device includes: a memory and a processor, the memory and the processor being coupled; the memory storing program instructions, which, when executed by the processor, cause the electronic device to execute the instructions of the method in the first aspect or any possible implementation of the first aspect, or to execute the instructions of the method in the second aspect or any possible implementation of the second aspect.

[0110] The third aspect and any implementation of the third aspect correspond to the first aspect and any implementation of the first aspect, and the second aspect and any implementation of the second aspect, respectively. The technical effects corresponding to the third aspect and any implementation of the third aspect can refer to the technical effects corresponding to the first aspect and any implementation of the first aspect, or refer to the technical effects corresponding to the second aspect and any implementation of the second aspect, and will not be repeated here.

[0111] In a fourth aspect, the present application provides a computer-readable medium for storing a computer program, which includes instructions for executing the method in the first aspect or any possible implementation of the first aspect, or instructions for executing the method in the second aspect or any possible implementation of the second aspect.

[0112] The fourth aspect and any implementation of the fourth aspect correspond to the first aspect and any implementation of the first aspect, and the second aspect and any implementation of the second aspect, respectively. The technical effects corresponding to the fourth aspect and any implementation of the fourth aspect can refer to the technical effects corresponding to the first aspect and any implementation of the first aspect, or refer to the technical effects corresponding to the second aspect and any implementation of the second aspect, and will not be repeated here.

[0113] In a fifth aspect, the present application provides a computer program comprising instructions for executing the method of the first aspect or any possible implementation of the first aspect, or instructions for executing the method of the second aspect or any possible implementation of the second aspect.

[0114] The fifth aspect and any implementation of the fifth aspect correspond to the first aspect and any implementation of the first aspect, and the second aspect and any implementation of the second aspect, respectively. The technical effects corresponding to the fifth aspect and any implementation of the fifth aspect can refer to the technical effects corresponding to the first aspect and any implementation of the first aspect, or refer to the technical effects corresponding to the second aspect and any implementation of the second aspect, and will not be repeated here.

[0115] In a sixth aspect, the present application provides a chip comprising a processing circuit and a transceiver pin. The transceiver pin and the processing circuit communicate with each other via an internal connection path, and the processing circuit executes the method of the first aspect or any possible implementation of the first aspect, or executes the method of the second aspect or any possible implementation of the second aspect, to control the receive pin to receive a signal and to control the transmit pin to send a signal.

[0116] The sixth aspect and any implementation of the sixth aspect correspond to the first aspect and any implementation of the first aspect, and the second aspect and any implementation of the second aspect, respectively. The technical effects corresponding to the sixth aspect and any implementation of the sixth aspect can refer to the technical effects corresponding to the first aspect and any implementation of the first aspect, or refer to the technical effects corresponding to the second aspect and any implementation of the second aspect, and will not be repeated here.

[0117] In a seventh aspect, the present application provides a system, comprising a first electronic device and a second electronic device, wherein the operating system of the first electronic device is a Windows system, and the operating system of the second electronic device is an Android system;

[0118] Among them, the first electronic device and the second electronic device can execute the method in the first aspect or any possible implementation of the first aspect, or execute the method in the second aspect or any possible implementation of the second aspect to realize cross-device dragging of windows.

[0119] The seventh aspect and any implementation of the seventh aspect correspond to the first aspect and any implementation of the first aspect, and the second aspect and any implementation of the second aspect, respectively. The technical effects corresponding to the seventh aspect and any implementation of the seventh aspect can refer to the technical effects corresponding to the first aspect and any implementation of the first aspect, or refer to the technical effects corresponding to the second aspect and any implementation of the second aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0120] 1A to 1D are schematic diagrams illustrating exemplary document dragging scenarios in a keyboard and mouse sharing scenario;

[0121] 2A to 2C are schematic diagrams illustrating exemplary window dragging scenarios in a keyboard and mouse sharing scenario;

[0122] 3A to 3E are schematic diagrams illustrating exemplary implementation of window dragging in a keyboard and mouse sharing scenario;

[0123] 4A to 4C are schematic diagrams illustrating sidebar styles corresponding to extended modes in different states of a floating window on an Android device after an extended connection is established between a Windows device and an Android device;

[0124] FIG5 is a schematic diagram showing an operation of a task bar corresponding to a control 50a-4 on a floating window 50a;

[0125] 6A and 6B are schematic diagrams showing exemplary reduction and enlargement of the floating window 50a;

[0126] FIG7 is a schematic diagram showing the change of the floating window 50a after the Android device is rotated from the landscape state to the portrait state;

[0127] 8A and 8B are schematic diagrams illustrating software architecture of a Windows device;

[0128] 9A and 9B are schematic diagrams illustrating exemplary software architectures of Android devices;

[0129] FIG10 is a timing diagram illustrating exemplary interactions between functional modules when a Windows device and an Android device implement the cross-device window dragging method provided in an embodiment of the present application;

[0130] FIG11 is a timing diagram exemplarily illustrating the interaction of functional modules involved when the Windows device in FIG10 implements the cross-device dragging method for a window provided in an embodiment of the present application;

[0131] FIG12 is a schematic diagram showing an exemplary relationship between a composite drawing canvas and a screen of a Windows device;

[0132] FIG13 is a flowchart showing an exemplary calculation of the hidden position of the window 20a;

[0133] 14A and 14B are schematic diagrams showing exemplary hidden positions of the window 20a;

[0134] FIG15 is a timing diagram exemplarily illustrating the interaction of functional modules involved when the Android device in FIG10 implements the cross-device dragging method for a window provided in an embodiment of the present application;

[0135] FIG16A is another timing diagram illustrating the interaction of functional modules involved when an Android device implements the cross-device window dragging method provided in an embodiment of the present application;

[0136] FIG16B is a schematic diagram illustrating an exemplary method of determining aspect ratios of extended screens of different sizes;

[0137] FIG17 is a schematic diagram illustrating interface changes on a Windows device and an Android device during the first dragging of a window 20a to an Android device, according to an exemplary method for dragging a window across devices provided by an embodiment of the present application, when the collaboration function is not enabled;

[0138] Figure 18 is an illustrative diagram of the cross-device dragging method of windows provided in an embodiment of the present application. When the collaborative function is started and the floating window 50a is displayed, the interface changes of the Windows device and the Android device are shown in the process of dragging the window 70a to the Android device. DETAILED DESCRIPTION

[0139] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0140] The term "and / or" in this article is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0141] In the description and claims of the embodiments of this application, the terms "first" and "second" are used to distinguish different objects, rather than to describe a specific order of objects. For example, the terms "first target object" and "second target object" are used to distinguish different objects, rather than to describe a specific order of objects.

[0142] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0143] In the description of the embodiments of this application, unless otherwise specified, "multiple" means two or more. For example, "multiple processing units" means two or more processing units; "multiple systems" means two or more systems.

[0144] With the popularization of electronic devices, more and more users own multiple (at least 2) electronic devices. In order to facilitate users to use multiple electronic devices for office and entertainment at the same time, the keyboard and mouse sharing function has been favored by more and more users. Based on the keyboard and mouse sharing function, electronic devices such as mobile phones and tablets that do not have or are not connected to input / output (I / O) devices such as keyboards and mice can share the use of computer keyboards, mice or touchpads.

[0145] Specifically in this application, electronic devices that do not have or are not connected to I / O devices specifically refer to Android devices (electronic devices whose operating system is the Android system), and computers specifically refer to Windows devices (electronic devices whose operating system is the Windows system).

[0146] In other words, Android devices and Windows devices with keyboard and mouse sharing capabilities can establish a keyboard and mouse connection, allowing the Windows device's mouse to move freely between the Android and Windows devices. In this way, users can use applications, view and edit files on different electronic devices by controlling the same mouse.

[0147] Regarding the establishment of a keyboard and mouse sharing connection, in some possible implementation methods, for example, the keyboard and mouse sharing connection can be established by placing the Android device that needs to establish a keyboard and mouse sharing connection next to the Windows device, and then moving the mouse pointer of the Windows device to the side edge of the screen of the Windows device and tapping the side edge of the screen of the Windows device twice in a row.

[0148] For example, in some other possible implementations, for example, you can place the Android device that needs to establish a keyboard and mouse sharing connection next to the Windows device, then enter the computer manager in the Windows device, and then establish a keyboard and mouse sharing connection between the Windows device and the Android device placed next to it through the keyboard and mouse sharing entrance under the Smart Interconnection directory in the computer manager.

[0149] It should be understood that the above description is merely an example for a better understanding of the technical solution of this embodiment and is not intended to be the sole limitation of this embodiment. For specific implementation details on establishing a shared keyboard and mouse connection between an Android device and a Windows device, please refer to the existing keyboard and mouse sharing documentation and will not be repeated here.

[0150] Furthermore, it should be noted that after establishing a keyboard and mouse sharing connection between an Android device and a Windows device, that is, when the Android and Windows devices are in a keyboard and mouse sharing scenario, the Windows device's mouse can not only move freely between the Android and Windows devices, but in some possible implementation scenarios, the user can also control the Windows device's mouse to drag and drop images and documents between different electronic devices. For example, a user can use the Windows device's mouse to drag and drop a document from a Windows device to an Android device.

[0151] In addition, it should be noted that after completing the above-mentioned drag and drop transfer of the document, the document exists not only in the Windows device, but also in the Android device, so that the user can view the document in both the Window device and the Android device.

[0152] In order to better understand the above-mentioned drag and drop transmission of documents using a mouse in a Windows device, a detailed description is given below with reference to FIG. 1A to FIG. 1D .

[0153] Referring to Figure 1A , this embodiment uses a Windows device as a PC, and the Android devices establishing a keyboard and mouse sharing connection with the PC as a tablet and a foldable screen phone, respectively. As shown in Figure 1A , when a user uses the PC's mouse to operate Document 2 placed on the PC desktop, for example, by moving the mouse cursor over Document 2 and pressing the left mouse button to drag Document 2, in the keyboard and mouse sharing scenario, as the user presses the left mouse button to drag the Document 2 icon, the Document 2 icon will move with the mouse cursor, such as from the position shown in Figure 1A to the position shown in Figure 1B .

[0154] For example, if the user continues to hold down the left mouse button and drag the Document 2 icon, when the mouse cursor moves to the side edge of the PC screen, such as the right edge of the screen in Figure 1C , Document 2 will be transferred to the tablet through the keyboard and mouse sharing connection between the PC and tablet. That is, the Document 2 icon will be displayed on the tablet screen, as shown in Figure 1D . At the same time, the Document 2 icon will continue to be displayed on the PC screen.

[0155] That is, currently, in a keyboard and mouse sharing scenario, users can only drag and drop images and documents between different electronic devices by operating the same mouse. If a user wants to drag an already open document, they must first close the document window using the close control in the title bar of the document window, and then use the drag and drop method shown in Figures 1A to 1D to drag the document to another electronic device. In other words, it is impossible to drag a currently open window from a Windows device to an Android device, that is, it is impossible to drag a window across devices.

[0156] In view of this, the present application provides a method for dragging windows across devices. Based on the method for dragging windows across devices provided by the present application, it is possible to drag windows across devices in a keyboard and mouse sharing scenario, especially to drag windows from a Windows device to an Android device.

[0157] Regarding the cross-device dragging of windows mentioned in this application, the changes presented on the screens of Windows devices and Android devices are shown in Figures 2A to 2C.

[0158] Referring to Figure 2A , for example, a Windows device is used as a PC, and an Android device that establishes a keyboard and mouse sharing connection with the PC is used as a tablet. In the keyboard and mouse sharing scenario shown in Figure 2A , when a user operates the PC mouse to open Document 2 on the PC, moves the mouse cursor (the arrow-shaped mouse cursor shown in Figure 2A ) to the title bar of the window corresponding to Document 2, and presses the left mouse button to drag the mouse toward the right edge of the PC screen, the Document 2 window will move toward the right edge of the PC screen along with the mouse cursor.

[0159] Referring to FIG2B , for example, when the window of Document 2 moves to the right edge of the PC screen along with the mouse cursor, if the user continues to press the left mouse button and move the cursor to the right, based on the cross-device dragging method for windows provided in this application, the window of Document 2 on the PC will be dragged to the screen of the tablet located to the right of the PC and with which a keyboard and mouse sharing connection has been established, as shown in FIG2C . That is, based on the cross-device dragging method for windows provided in this application, the window of Document 2 displayed on the PC will be displayed as a floating window on the screen of the tablet, and the window of Document 2 will no longer be displayed on the screen of the PC.

[0160] Continuing with Figure 2C , for example, after the document 2 window is moved from the PC to the tablet, the mouse connected to the PC will also be moved to the tablet. That is, the mouse cursor will move from the PC to the tablet. In one possible implementation, after the mouse cursor moves from the PC to the tablet, the displayed mouse cursor may be the dot-style cursor shown in Figure 2C . Furthermore, when the mouse cursor first moves from the PC to the tablet, it may be located on the status bar 1 of the floating window displayed on the tablet.

[0161] Regarding the switching of styles of the mouse cursor at different positions, the setting of the initial position, and the functions of the status bar 1 and the status bar 2 shown in FIG. 3A , please refer to the following embodiments for details and will not be described in detail here.

[0162] Thus, when a keyboard and mouse sharing connection is established between a Windows device and an Android device, that is, when the device is in a keyboard and mouse sharing scenario, when a user uses a Windows device, such as the PC mentioned in the above embodiment to open multiple applications, and needs to use these applications at the same time, such as when viewing the contents in the windows corresponding to these applications, the cross-device dragging method of the window provided by this application can be performed by dragging and monitoring the window corresponding to the application / document on the PC, and cooperating with the mouse crossing control in the keyboard and mouse sharing. When the mouse crosses, the position where the window moves out of the PC screen is calculated, and the information related to the window is transmitted to the Android device, such as the tablet mentioned in the above embodiment. When the window is released on the tablet (the user releases the left mouse button), a floating window will be started on the tablet as an extended screen of the PC, and the window of document 2 as mentioned in the above embodiment will be displayed in the floating window, and the window of document 2 will no longer be displayed on the PC, thereby realizing the experience of dragging the window on the PC to the tablet for display.

[0163] In addition, it should be noted that the cross-device dragging method for windows provided in this application is to enable Windows devices in keyboard and mouse sharing scenarios to achieve dual-screen or even multi-screen (more than 3) office experiences with the help of Android devices. That is, the window dragged by the mouse also needs to be displayed in the form of a window. Therefore, the Android device mentioned in this application can be an Android device with a relatively large screen, such as an Android tablet, an Android folding screen mobile phone, etc. In this way, the currently used window can be displayed in a tiled manner, making it easier for the user to view the content in the window.

[0164] In addition, it should be noted that in order to enable the tablet to serve as an extended screen of the PC, and then in the keyboard and mouse sharing scenario, the window in the PC can be dragged and dropped to the tablet for display using the mouse. When implementing the cross-device window dragging method provided in this application, the screen collaboration function (hereinafter referred to as the collaboration function) is also required.

[0165] Specifically in each embodiment of the present application, the collaborative function includes: through distributed technology, collaboratively connecting devices of different systems, and then using the Sink end (receiving end) as an extended screen of the Source end (sending end), thereby realizing a multi-screen office experience.

[0166] Based on the collaborative function, users can achieve a multi-screen experience (at least two screens). For example, multiple files (documents, pictures, etc.) on the source side, such as file A and file B, can be opened on the source side and file B on the sink side, so that files A and B can be displayed side by side, making it convenient for users to view them simultaneously.

[0167] It should be understood that the above description is merely an example listed for a better understanding of the technical solution of this embodiment, and is not intended to be the sole limitation on this embodiment.

[0168] In addition, it should be noted that the collaborative function can be divided into mirror mode and extended mode. The so-called mirror mode can be understood as the content displayed in the floating window displayed on the tablet is exactly the same as that on the PC screen. The so-called extended mode can be understood as the content not displayed on the PC screen is expanded in the floating window displayed on the tablet.

[0169] Based on this, in order to achieve the effect of dragging the window corresponding to document 2 on the PC to be displayed on the tablet, this application adopts the extension mode in the collaborative mode.

[0170] To better understand the process of dragging a window from a Windows device, establishing a collaborative connection between the Android device and the Windows device, starting the extended mode of the collaborative function, and then displaying the window dragged from the Windows device on the Android device screen, the following detailed description of the changes on the screens of the Windows device and the Android device during this process is provided with reference to the accompanying figures.

[0171] Still taking the example of the Windows device mentioned in the above embodiment as a PC, the window to be dragged as the window of Document 2 displayed on the PC, and the Android device to be dragged as a tablet, for ease of explanation, in this embodiment, the arrow-shaped cursor corresponding to the mouse on the PC is referred to as cursor 10a, the dot-shaped cursor corresponding to the tablet after the mouse is moved to the tablet is referred to as cursor 10b, the window corresponding to Document 2 on the PC is referred to as window 20a, the window corresponding to Document 2 on the tablet is referred to as window 20b, the taskbar displayed at the bottom of the PC screen is referred to as taskbar 30a, the floating window displayed before the collaborative function is started after the mouse is moved to the tablet is referred to as floating window 40a, the floating window (equivalent to the extended screen of the PC) that replaces floating window 40a after the collaborative function is started is referred to as floating window 50a, the taskbar 30a displayed in floating window 50a is referred to as taskbar 30b, the status bars at the top of floating window 40a and floating window 50a are both referred to as status bar 1, and the status bars at the bottom are both referred to as status bar 2.

[0172] It should be understood that the above description is merely an example listed for a better understanding of the technical solution of this embodiment, and is not intended to be the sole limitation on this embodiment.

[0173] In addition, it should be noted that in the keyboard and mouse sharing scenarios described in the various embodiments of this application, the mouse that can be shared by Windows devices and Android devices is exemplified by the mouse connected to the PC device shown in Figures 1A to 1D. The mouse is no longer shown in subsequent figures.

[0174] For example, when the user holds down the left mouse button and drags window 20a to the right edge of the PC screen (as shown in FIG2B ), if the user does not release the left mouse button and continues to hold down the left mouse button and drag window 20a to the right, cursor 10a will gradually approach the right edge of the PC screen. To achieve the effect of window 20a crossing from the PC to the tablet, as cursor 10a gradually approaches the right edge of the PC screen, window 20a will be partially moved off the PC screen, that is, only the portion before the position of cursor 10a is displayed on the PC screen, as shown in FIG3A . Since, in FIG3A , cursor 10a has not yet moved to the right edge of the PC screen, that is, the mouse has not yet crossed, cursor 10b and window 20a will not be displayed on the tablet.

[0175] As shown in Figure 3B , for example, the user presses the left mouse button and continues to drag window 20a. When cursor 10a moves to the right edge of the PC screen, the PC's mouse cursor is transferred from the PC to the tablet, based on the shared keyboard and mouse connection established between the PC and tablet. That is, cursor 10a transfers to the tablet and transforms into cursor 10b. Simultaneously, a floating window 40a, approximately one-third the size of the original window, appears on the tablet, while window 20a on the PC is hidden, as shown in Figure 3C .

[0176] It should be noted that before the user releases the left mouse button to activate the collaboration function and displays the floating window 50a as an extended screen of the PC on the tablet, the user can move the mouse back to the PC at any time, and the dragged window 20a will return to the PC screen. Therefore, before the collaboration function is activated, in order to visually make the user feel that window 20a has moved from the PC to the tablet, a floating window 40a will be displayed first. In other words, floating window 40a is not an extended screen of the PC, but only a floating window drawn and displayed on the tablet. After the floating window 50a as an extended screen of the PC is activated, the floating window 50a will cover the floating window 40a, and the floating window 40a will be destroyed.

[0177] Based on this, in order to prevent the user from feeling that the floating window 40a is replaced by the floating window 50a, in some possible cases, the floating window 40a and the floating window 50a can be drawn as the same size, and the status bar 1 and the status bar 2 have the same function as the selection controls.

[0178] This embodiment is described by taking the example of setting function controls in the status bar 1 , but in actual applications, function controls may also be set in the status bar 2 according to business requirements.

[0179] As shown in Figures 3D and 3E , control 40a-1 in floating window 40a and control 50a-1 in floating window 50a function identically: both are used to close the floating window currently displayed on the tablet screen. Since the collaboration feature is already enabled, clicking control 50a-1 disconnects the collaboration between the tablet and the PC, effectively disabling the tablet from functioning as an extended PC screen.

[0180] Continuing with Figures 3D and 3E , control 40a-2 in floating window 40a functions the same as control 50a-2 in floating window 50a, both for maximizing the floating window currently displayed on the tablet screen. Control 40a-3 in floating window 40a functions the same as control 50a-3 in floating window 50a, both for minimizing the floating window currently displayed on the tablet screen. Control 40a-4 in floating window 40a functions the same as control 50a-4 in floating window 50a, both for popping up a hidden taskbar at the top edge of the current floating window.

[0181] Continuing with FIG3D , for example, if the user continues to hold down the left mouse button to drag the floating window 40a until the floating window 40a is fully displayed on the tablet screen, the user releases the button. When the mouse is released, the PC sends an instruction to release the window 40a to the tablet based on the keyboard and mouse sharing connection. After receiving the instruction to release the window 40a, the tablet releases the window 40a. The operation of releasing the window 40a can be understood as the tablet injecting an up event into the mouse. The specific implementation details of this operation can be found in the description of step 221 in the following embodiment, which will not be repeated here.

[0182] When the mouse is released and floating window 40a is displayed on the tablet, the PC activates the collaborative function and establishes a collaborative connection with the tablet in extended mode. After the collaborative connection is established, the tablet creates floating window 50a, decompresses the collaborative screen transmitted by the PC via the collaborative connection, and then displays the collaborative screen in the display area of ​​floating window 50a.

[0183] The display area of ​​the floating window 50a is the area of ​​the floating window 50a excluding the status bar 1 and the status bar 2.

[0184] In addition, since the floating window 50a is used as an extended screen of the PC, the taskbar 30a of the PC will be displayed in the floating window 50a. Therefore, the collaborative screen needs to include the window 20b and the taskbar 30b.

[0185] In addition, in order to give users a better user experience, in one possible implementation method, the collaborative screen can be set to occupy the entire display area of ​​the floating window 50a. In this way, the user visually sees that window 20a is dragged to the tablet as a whole. Therefore, when generating the collaborative screen, the synthesis (drawing) rendering module in the PC specifically draws window 20b based on window 20a in maximized mode. That is, window 20b is essentially window 20a in maximized mode.

[0186] For example, for the window 20a that is not maximized in this embodiment, the functional controls displayed in its title bar include a maximize control for maximizing the window 20a, as shown in FIG2A . For the window 20b drawn based on the window 20a in maximized mode, since it is essentially the window 20a in maximized mode. Therefore, in each embodiment of the present application, the functional controls displayed in the title bar of the window 20b displayed in the floating window 50a dragged to the tablet will not include a maximize control, but will be replaced by a restore control, as shown in FIG3C .

[0187] In this way, after the floating window 40a is covered by the floating window 50a, the user can directly see the window 20b, as shown in FIG3E .

[0188] Continuing to refer to FIG3D , for example, while waiting for the PC to establish a collaborative connection with the tablet and using the floating window 50a to cover the floating window 40a, a dynamic loading icon, such as the icon 40a-5 shown in FIG3D , may also be displayed in the floating window 40a to prompt the user that the window 20a is currently being loaded.

[0189] It should be understood that the above description is merely an example listed for a better understanding of the technical solution of this embodiment, and is not intended to be the sole limitation on this embodiment.

[0190] Referring to FIG3E , for example, due to the cross-device dragging solution for windows provided in this embodiment, the entire screen of the tablet is not used as an extended screen of the PC. Instead, a floating window 50a is first displayed on the tablet, and the display area within the floating window 50a is used as an extended screen of the PC. Therefore, in order not to block the content in the window 20a in the floating window 50a, the sidebar that would be displayed at the edge of the extended screen (usually at the left / right edge of the screen for tablets and at the top / bottom edge of the screen for foldable screen phones) in the extended mode of the collaborative function will be hidden. That is, the sidebar is not displayed in the floating window 50a.

[0191] For example, in some possible implementations, a hot zone can be set on the right edge of the floating window 50a. When the user moves the cursor 10b to this area, presses the left mouse button to slide to the left, or clicks / double-clicks the left mouse button, and then the hidden sidebar is displayed on the right edge of the floating window 50a.

[0192] For example, in some other possible implementations, an operable control may be provided on the right edge of the floating window 50a, such as control 50a-5 shown in FIG3E . When the user moves the cursor 10b to control 50a-5 and clicks the left mouse button, the hidden sidebar may be displayed on the right edge of the floating window 50a.

[0193] It should be understood that the above description is merely an example for better understanding the technical solution of this embodiment and is not intended to be the sole limitation of this embodiment. For ease of explanation, the hiding and displaying of the sidebar is described in this embodiment in the form of the control 50a-5 shown in FIG3E.

[0194] From the above definition of cursor styles, we can see that when the cursor is acting on a PC, its style is the style of cursor 10a, and when the cursor is acting on a tablet, its style is the style of cursor 10b. Since floating window 50a serves as an extended screen of the PC, when the cursor moves to the display area inside floating window 50a, that is, the area where window 20b and taskbar 30b are displayed in Figure 4A, it is equivalent to acting on the PC, and the cursor will switch from the style of cursor 10b to the style of cursor 10a.

[0195] Referring to (1) in FIG. 4A , when the cursor 10a moves onto the control 50a - 5 and the user clicks the left mouse button, the control 50a - 5 disappears and is replaced by a sidebar 50a - 6 as shown in (2) in FIG. 4A .

[0196] It is understandable that the sidebar 50a-6 will display function options related to the extended mode, which are not shown in the sidebar 50a-6 due to the size of the area (2) in Figure 4A. To better understand the function options included in the sidebar, the following is an explanation in conjunction with Figure 4B.

[0197] Referring to Figure 4B, for example, as an optional situation, the sidebar 50a-6 may include a function option for returning to the desktop, a function option for saving the content in the extended screen, a function option for undoing the last operation, a function option for moving the current window to the opposite end display, a function option for switching the collaborative mode, a function option for port collaboration, etc., which are not listed one by one here, and this embodiment does not limit this.

[0198] Understandably, in one possible implementation, the sidebar 50a-6 can be set to be undisplayed after any function option in the sidebar 50a-6 is selected. In another possible implementation, the sidebar 50a-6 can also be set to automatically disappear after being displayed for a set period of time, at which time the control 50a-5 can be redisplayed in the floating window 50.

[0199] It should be understood that the above description is merely an example listed for a better understanding of the technical solution of this embodiment, and is not intended to be the sole limitation on this embodiment.

[0200] In addition, regarding the function option for switching collaborative modes, if this function option is selected while currently in extended mode, the collaborative mode will switch from extended mode to mirror mode. Conversely, if this function option is selected while currently in mirror mode, the collaborative mode will switch from mirror mode to extended mode. Due to the difference between mirror mode and extended mode, the function options displayed in the sidebar will also change. The specific function options may be determined based on the two collaborative modes, and this embodiment does not impose any restrictions on this.

[0201] Continuing with FIG3E , for example, if the user operates the mouse to move the cursor 10b to the control 50a-2 and clicks the left mouse button, the floating window 50a will be maximized and displayed on the tablet screen, as shown in FIG4C . In this case, the status bar 1 and status bar 2 of the floating window 50a will not be displayed, but in maximized mode, the left or right side of the tablet screen (the left side is used as an example in this embodiment) will display the sidebar corresponding to the current collaborative mode, such as the sidebar 50a-7 shown in FIG4C . Still taking the current collaborative mode as the extended mode as an example, the sidebar 50a-7 can display the same functional options as the sidebar 50a-6.

[0202] Since window 20b in the collaborative screen is drawn based on the maximized window 20a, when floating window 50a is maximized, the collaborative screen received from the PC will also be enlarged as a whole. Therefore, when floating window 50a is maximized, window 20b and taskbar 30b will still occupy the entire display area of ​​floating window 50a, as shown in Figure 4C.

[0203] In addition, operations on the floating window 50a can also be performed through the function options in the taskbar corresponding to the control 50a-4. For example, when the cursor 10b moves to the control 50a-4 and the user clicks the left mouse button, a hidden taskbar can pop up in the area where the control 50a-4 is located, such as the taskbar 50a-8 shown in (1) of Figure 5.

[0204] Referring to (1) in Figure 5, for example, in some possible implementations, the taskbar 50a-8 may include the same functional controls as controls 50a-1, 50a-2, and 50a-3, as well as functional options for switching collaborative modes shown in the sidebar 50a-6 or sidebar 50a-7.

[0205] For example, when the user clicks on the same functional control as the control 50a-2 in the taskbar 50a-8, the floating window 50a will be maximized, that is, the tablet displays the interface content as shown in (2) in Figure 4.

[0206] For example, when the user clicks the same function control as control 50a-3 in taskbar 50a-8, floating window 50a will be minimized. In one possible implementation, the minimized floating window will be displayed in the style of capsule 60a shown in (2) of Figure 5. When the user clicks capsule 60a, floating window 50a will be redisplayed in the tablet interface, for example, in the style shown in (1) of Figure 4.

[0207] For example, when the user clicks the same function control as control 50a-1 in taskbar 50a-8, the tablet will disconnect from the PC and close the floating window 50a. In other words, the tablet will return to the state before the window 20 was dragged to the tablet, as shown in FIG3B.

[0208] It should be noted that when the collaborative connection between the tablet and the PC is disconnected, in some possible implementations, the PC can automatically display window 20a. In other possible implementations, the user can click a specified location or control to redisplay window 20a on the PC screen.

[0209] It should be understood that the above description is merely an example listed for a better understanding of the technical solution of this embodiment, and is not intended to be the sole limitation on this embodiment.

[0210] For example, when a user clicks the collaborative mode switch option in taskbar 50a-8, the collaborative mode is switched, and the content displayed in floating window 50a changes accordingly. For example, when switching to mirror mode, floating window 50a will display the current desktop of the PC. When switching back to extended mode, the window dragged into floating window 50a will be displayed, such as window 20a described above.

[0211] From the above description, it can be seen that the taskbar 50a-8 will be displayed after the control 50a-4 is clicked, and the control 40a-4 in the floating window 40a has the same function as the control 50a-4. Therefore, in order to avoid the cursor 10b displayed on the tablet interface clicking on the control 40a-4 when the mouse just crosses from the PC to the tablet, when the cursor 10a crosses from the PC to the tablet, the cursor 10b that appears in the tablet can be displayed in the status bar 1 of the floating window 40a, and at a position away from the control 40a-4, as shown in Figure 3C.

[0212] It should be understood that the above description is merely an example listed for a better understanding of the technical solution of this embodiment, and is not intended to be the sole limitation on this embodiment.

[0213] In addition, it should be noted that, in this embodiment, the user can also zoom in or out the floating window 50a displayed on the tablet by manipulating the mouse.

[0214] Exemplarily, when the user moves the cursor to the edge of the floating window 50a, that is, removes the display area of ​​the floating window 50a, the cursor style will be switched back to the cursor 10b.

[0215] Referring to (1) in FIG. 6A , for example, when the cursor 10b moves to the lower right corner of the floating window 50a, the user presses the left mouse button and drags the floating window 50a in the direction of the arrow, and the floating window 50a is reduced in size. After dragging to the position shown in (2) in FIG. 6A , the user releases the left mouse button, and the floating window 50a is reduced to the size shown in (2) in FIG. 6A and remains at that position.

[0216] Referring to (1) in FIG. 6B , for example, when the cursor 10b moves to the lower right corner of the floating window 50a, the user presses the left mouse button and drags the floating window 50a in the direction of the arrow, and the floating window 50a is enlarged as a whole. After dragging to the position shown in (2) in FIG. 6B , the user releases the left mouse button, and the floating window 50a is enlarged to the size shown in (2) in FIG. 6B and remains at that position.

[0217] In addition, it should be noted that, in order to ensure the user experience, in the window dragging method provided in this embodiment, when the tablet displaying the floating window 50a is rotated from the horizontal screen state to the vertical screen state, such as rotating from (1) in 7 along the arrow direction to (2) in Figure 7, the floating window 50a will also rotate as a whole, and the size of the floating window 50a will be adjusted according to the horizontal distance after the vertical screen.

[0218] In addition, in some possible implementations, when the screen is rotated from landscape to portrait, or vice versa, the floating window 50a will be displayed centered on the left and right sides. However, when the user drags the floating window 50a, it will stay at the position where the user dragged it.

[0219] It should be understood that the above description is merely an example listed for a better understanding of the technical solution of this embodiment, and is not intended to be the sole limitation on this embodiment.

[0220] In order to better understand the cross-device dragging method for windows provided in this application, the software architecture of Windows devices and the software architecture of Android devices that implement the cross-device dragging method for windows are respectively described in conjunction with Figures 8A, 8B, 9A and 9B.

[0221] It should be noted that in practical applications, whether it is a Windows device or an Android device, its software system (operating system) can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. In addition, for both Windows and Android devices, they can be divided into user mode and kernel mode.

[0222] Referring to FIG8A , a software architecture of a Windows device is exemplarily shown.

[0223] As shown in Figure 8A, from the perspective of user state, from bottom to top, it can include NTDLL, service process, system process, environment subsystem, data transmission module, user process, etc. Specifically in this embodiment, the cross-device dragging of windows mainly depends on the keyboard and mouse sharing service and collaborative service in the service process, as well as the data transmission mode implementation. Among them, the keyboard and mouse sharing service (process) is a service process that implements the above-mentioned keyboard and mouse sharing function; the collaborative service (process) is a service process that implements the above-mentioned collaborative function, that is, it is responsible for establishing a collaborative connection between Windows devices and Android devices; the data transmission module is responsible for signaling and data transmission. In one implementation method, when both Windows devices and Android devices are electronic devices generated by Honor manufacturers, the data transmission module is, for example, MagicLink.

[0224] For the functions of other processes, systems, NTDLL, etc. in user mode, please refer to the introduction of the existing Windows system architecture, which will not be repeated here.

[0225] Furthermore, it should be noted that in some possible implementations, user processes may include processes corresponding to applications similar to the aforementioned PC Manager. The PC Manager's corresponding processes (e.g., service management processes) can be used to manage various services and schedule various virtualized services. For example, this can include scheduling keyboard and mouse sharing services and collaborative services within service processes.

[0226] In addition, it should be noted that since this embodiment realizes cross-device dragging of windows in the original keyboard and mouse sharing scenario, the keyboard and mouse sharing service is improved. The improved keyboard and mouse sharing service includes at least the four functional modules shown in Figure 8B.

[0227] 8B , illustratively, the keyboard and mouse sharing service in a Windows device includes a window management module, a system message module, a window setting module, and a protocol management module.

[0228] Among them, the window management module is responsible for the management of dragging windows, the management of window messages, position identification, etc.; the window setting module is responsible for the position calculation of window crossing, hiding, and display; the system message is responsible for the message of the mouse starting to drag the window or stopping to drag the window; the protocol management is responsible for the packaging and unpacking of data such as related commands, window parameters, etc. for window dragging.

[0229] It's important to note that in Windows programming, when each Windows application begins executing, the system creates a message queue for that application. This message queue is used to store information about the windows created by that application. For example, when a user clicks the mouse on a window, a WM_RBUTTONDOWN message is generated. The system automatically places this message into the message queue of the application to which the current window belongs, waiting for the application to terminate. Windows sequentially places these messages into the message queue, and the application continuously reads and responds to messages from the message queue through a message loop. When the mouse is moved or stopped, the system generates system messages indicating window movement, messages managed by the aforementioned system message module.

[0230] Continuing to refer to FIG8A , from the perspective of kernel state, the system may include, from bottom to top, the hardware abstraction layer, the kernel and device driver layer, the window management and image drawing module, and the Windows executive body. Specifically, in the technical solution provided in this embodiment, dragging windows across devices not only relies on the keyboard and mouse sharing function, but also on the collaboration function. Both of these functions rely on WiFi or Bluetooth, and therefore require functional modules related to WiFi and Bluetooth, such as WiFi drivers and Bluetooth drivers.

[0231] In addition, for Windows devices that do not have I / O devices, they also need to use USB interfaces, Bluetooth, or WiFi to connect external keyboards, mice, etc. Therefore, in addition to the functional modules related to WiFi and Bluetooth, they also need to involve functional modules related to USB, such as USB drivers.

[0232] For descriptions of the Windows executive in kernel state, other functional modules involved in the kernel and device driver layers, window management and drawing modules, hardware abstraction layer, etc., please refer to the introduction to the existing Windows system architecture and will not be repeated here.

[0233] This concludes the introduction to the software architecture of Windows devices. It should be understood that the layers and functional modules within each layer shown in Figures 8A and 8B do not constitute specific limitations on Windows devices. In some possible implementations, a Windows device may include more or fewer layers than shown, and each layer may include more or fewer components, and this application does not impose any limitations.

[0234] Referring to FIG. 9A , a software architecture of a Windows device is exemplarily shown.

[0235] As shown in Figure 9A, the layered architecture of an Android device divides the software into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some implementations, the Android system is divided into five layers: from top to bottom: the application layer (Application, APP layer), the framework layer (Application Framework, FWK layer), the Android runtime (Android runtime) and system libraries, the hardware abstraction layer (HAL layer), and the kernel layer (Linux kernel, kernel layer).

[0236] The APP layer may include a series of application packages. As shown in FIG9A , the application package may include camera, game, video, live broadcast and other applications, which are not listed here one by one and are not limited in this application.

[0237] The FWK layer provides application programming interfaces (APIs) and programming frameworks for applications in the APP layer. In some implementations, these programming interfaces and programming frameworks can be described as functions. As shown in Figure 9A, the FWK layer may include functions such as content providers, keyboard and mouse sharing services, collaborative services, and data transmission modules. These functions are not listed here and are not limited in this application.

[0238] The functions of the collaborative service and data transmission modules in Android devices are the same as those in Windows devices and will not be repeated here.

[0239] The keyboard and mouse sharing service on Android devices differs from the keyboard and mouse sharing service on Windows devices. Its main functional modules, as shown in Figure 9B, include a window management module, a window settings module, a protocol management module, and a virtual data injection module. The functions of these modules are identical to those on Windows devices and are not further described here.

[0240] Since the user did not release the left mouse button and then press it again when the mouse crossed from the Windows device to the Android device, the cursor 10b displayed on the status bar 1 of the floating window 40a (as shown in Figure 3B) does not have a corresponding down event on the tablet side. In this way, when the user releases the mouse and the PC sends a window release instruction to the tablet, the tablet cannot detect the up event and thus release the floating window 40a. In order to enable the tablet to release the floating window 40a, in this embodiment, the keyboard and mouse sharing service in the tablet will actively register a down event through the virtual data injection module when the mouse crosses. In this way, when the user subsequently releases the left mouse button, an up event will be detected.

[0241] 9A , for example, the Android Runtime includes a core library and a virtual machine. The Android Runtime is responsible for scheduling and management of the Android system.

[0242] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the Android core library.

[0243] The app and framework layers run in a virtual machine. The virtual machine executes Java files from the application and framework layers as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0244] Continuing to refer to FIG9A , illustratively, the system library may include multiple functional modules, such as a surface manager, a media library, a three-dimensional (3D) graphics processing library (e.g., OpenGL ES), a two-dimensional (2D) graphics engine (e.g., SGL), etc.

[0245] The surface manager manages the display subsystem and provides fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. The media library supports a variety of audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG. The 3D graphics processing library implements 3D graphics drawing, image rendering, compositing, and layer processing.

[0246] It can be understood that the 2D graphics engine mentioned above is a drawing engine for 2D drawing.

[0247] Continuing to refer to FIG9A , illustratively, the HAL layer is an interface layer located between the operating system kernel (kernel layer) and the hardware circuit. Its purpose is to isolate the FWK layer from the kernel so that Android does not become overly dependent on the kernel, thereby allowing the development of FWK to be carried out without considering the driver.

[0248] Continuing to refer to FIG9A , illustratively, the HAL layer may include various interfaces, such as audio and video interfaces, USB interfaces, Bluetooth interfaces, WiFi interfaces, etc., which are not listed here one by one and are not limited in this application.

[0249] 9A , the kernel layer in the Android system is illustratively a layer between hardware and software. The kernel layer may include various processes / threads, a power management module, and various drivers, such as a WiFi driver, a Bluetooth driver, and a kernel driver (Uinput driver) for simulating user input events.

[0250] This concludes the introduction to the software architecture of Android devices. It should be understood that the layers and functional modules within each layer shown in Figures 9A and 9B do not constitute specific limitations on Android devices. In some possible implementations, an Android device may include more or fewer layers than shown, and each layer may include more or fewer components, and this application does not impose any limitations.

[0251] In order to better understand the specific implementation of the cross-device dragging method for windows provided by this application, the following, in combination with the accompanying drawings, describes in detail the interaction logic of the keyboard and mouse sharing service (including functional modules), collaborative services, and data transmission modules in Windows devices, and the keyboard and mouse sharing service (including functional modules), collaborative services, and data transmission modules in Android devices.

[0252] Referring to Figure 10 , to implement the cross-device window dragging method provided in the application, the system message module, window management module, window setting module, protocol management module, collaborative service, and data transmission module in the Windows device must cooperate with each other and perform interactive processing in the order of steps 101 to 139 shown in Figure 10 . The data transmission module, protocol management module, window setting module, virtual data injection module, window setting module, and collaborative service in the Android device must then perform interactive processing in the order of steps 201 to 227 shown in Figure 10 . For the sequential logic between the operations processed by the Windows device and the operations processed by the Android device, please refer to Figure 10 .

[0253] To facilitate the distinction between functional modules in Windows devices and Android devices, in the following embodiments, the system message module, window management module, window setting module, protocol management module, collaborative service, and data transmission module in Windows devices are respectively represented as the Windows system message module, Windows window management module, Windows window setting module, Windows protocol management module, Windows collaborative service, and Windows data transmission module; the window management module, window setting module, protocol management module, collaborative service, data transmission module, and virtual data injection module in Android devices are respectively represented as the Android window management module, Android window setting module, Android protocol management module, Android collaborative service, Android data transmission module, and Android virtual data injection module.

[0254] The specific details of steps 101 to 139 implemented by the Windows device shown in Figure 10 are described in detail below in conjunction with Figure 11. It should be noted that Figure 11 only shows part of the text, and the specific details can be found in the corresponding text part.

[0255] Referring to FIG. 11 , to implement the cross-device window dragging method provided by the application, the system message module, window management module, window setting module, protocol management module, collaborative service, and data transmission module in the Windows device execute steps 101 to 139, specifically including:

[0256] 101. The Windows window management module sends a request for registering a window movement message fence to the Windows system message module in advance, so that the Windows message system can monitor the movement of windows displayed in the Windows device.

[0257] 102. The Windows window management module sends a request for registering a mouse message fence to the Windows system message module in advance, so that the Windows message system can monitor the mouse used in the keyboard and mouse sharing scenario.

[0258] Specifically, in this embodiment, dragging a window across devices relies on the keyboard and mouse sharing function. That is, the Android device to which the window is dragged from the Windows device has established a keyboard and mouse sharing connection with the Windows device. Therefore, in one feasible implementation, the operations of steps 101 and 102 can be triggered after the keyboard and mouse sharing connection is established between the Windows device and the Android device.

[0259] For example, when a user uses a mouse in a keyboard and mouse sharing scenario to drag a window displayed on a Windows device (for example, window 20a in the above embodiment), the window move message fence and mouse message fence registered in steps 101 and 102 are triggered. In this case, the Windows system message module executes steps 103 and 104.

[0260] 103. The Windows system message module obtains a window move message corresponding to the window 20a, and sends the window move message to the Windows window management module.

[0261] 104. The Windows system message module obtains a message indicating that the mouse used in the keyboard and mouse sharing scenario is pressed, determines that the left button of the mouse is currently pressed, and sends the left button pressed message to the window management module of the Windows device.

[0262] 105. After receiving both the window move message and the mouse left button press message sent by the Windows system message module, the Windows window management module determines that the window 20a is currently being dragged and moved.

[0263] It should be noted that in some possible implementations, the dragged window 20a may be displayed in maximized mode on the Windows device, i.e., the window 20a occupies the entire display area of ​​the Windows device. In this case, upon confirming that the window 20a is being dragged, the Windows device automatically restores the window 20a from maximized mode to small window mode in response to the drag operation, and then moves the window 20a as the mouse is dragged.

[0264] In addition, it should be noted that in some other possible implementations, the dragged window 20a may be currently displayed in small window mode, such as the style shown in Figure 2A. In this case, when it is confirmed that the window 20a is being dragged and moved, the window 20a maintains the current style and can be directly moved with the mouse.

[0265] Furthermore, it should be noted that since the Windows device and the Android device are currently sharing a keyboard and mouse, when the user uses the mouse to drag window 20a, it is possible that the user intends to drag window 20a displayed on the Windows device to the Android device for display. Therefore, when the Windows window management module determines that window 20a is being dragged, it sends the window parameters of window 20a to the Windows window setting module for subsequent processing, i.e., executing step 106.

[0266] Furthermore, it should be noted that, in one possible scenario, the user may use the arrow keys and shortcut keys on the keyboard to control the movement of window 20a on the screen of the Windows device. In this case, since the user does not operate the mouse, only the window move message fence registered in step 101 is triggered. Therefore, the Windows system message module only executes step 103, which sends the window move message to the Windows window management module for processing. Since there is only a window move message and no left mouse button press message, the Windows window management module determines that window 20a is not currently being dragged and moved. In this case, the Windows window management module will not execute step 106.

[0267] In another possible scenario, the user may have simply pressed the left mouse button to click a control in window 20a, or clicked another icon or document on the desktop. In this case, since window 20a has not moved, the Windows system message module only executes step 104, sending the left mouse button press message to the Windows window management module for processing. Since there is only a left mouse button press message and no window move message corresponding to window 20a, the Windows window management module determines that window 20a is not currently being dragged. In this case, the Windows window management module does not execute step 106.

[0268] In another possible scenario, the user may have pressed the right mouse button. In this case, since window 20a has not moved, the Windows system message module only executes step 104, sending the right mouse button press message to the Windows window management module for processing. Since there is only a right mouse button press message, no left mouse button press message, and no window move message corresponding to window 20a, the Windows window management module determines that window 20a is not currently being dragged and moved. In this case, the Windows window management module will not execute step 106.

[0269] It should be understood that the above description is merely an example listed for a better understanding of the technical solution of this embodiment, and is not intended to be the sole limitation on this embodiment.

[0270] 106. The Windows window management module sends the relevant window parameters of the window 20a to the Windows window setting module.

[0271] The window parameters of window 20a include, for example, the handle of window 20a. As can be understood, the window handle includes the coordinates of the starting point (vertex) of the upper left corner of the window, as well as the width and height of the window. Therefore, by obtaining the handle of window 20a, the size information of window 20a (width data xWidth, height data yHeight) can be determined.

[0272] 107. The Windows window setting module calculates the window data (such as width data xWidth and height data yHeight) of the window 20a according to the window parameters of the window 20a sent by the Windows window management module.

[0273] As can be seen from the description of the above embodiment, in the cross-device dragging method for windows provided in this application, the crossing of window 20a occurs when the mouse crosses. Therefore, after the Windows system message module monitors the window move message and the left mouse button press message of window 20a, if the user continues to press the left mouse button to drag window 20a, such as moving window 20a to the side edge of the screen, such as the right edge in the above embodiment, as the cursor displayed on the Windows device, such as the cursor 10a in the above embodiment, moves, the Windows system message module will continue to obtain the mouse movement position information and send the mouse movement position information to the Windows window management module, that is, execute step 108.

[0274] It should be noted that, in this embodiment and the following embodiments, the mouse movement position information refers to the cursor corresponding to the mouse, such as the movement position information of the cursor 10a in the above embodiment.

[0275] 108. The Windows system message module sends the mouse movement position message to the Windows window management module.

[0276] 109. The Windows window management module performs edge detection based on the mouse movement position information to determine whether the mouse has been dragged to the side edge of the Windows device screen (such as the right edge in the above embodiment).

[0277] Understandably, for Windows devices, the screen size is known. Therefore, when the screen size is known and the current mouse movement position information (coordinates) of the cursor 10a is also known, it is possible to determine whether the mouse has been dragged to the side edge of the Windows device screen.

[0278] Specifically, in a keyboard and mouse sharing scenario, when the Windows window management module determines that the mouse has been dragged to the edge of the screen (left edge or right edge), it triggers the keyboard and mouse to cross. At the same time, in the cross-device window dragging solution provided by this application, the Windows window management module notifies the Windows window settings module that the drag cross has occurred, that is, executes step 110, and then enables the Windows window settings module to generate a window cross instruction (command) based on the window data, that is, executes step 111.

[0279] 110. When the Windows window management module determines that the mouse has reached the edge of the screen based on the mouse movement position information, the Windows window setting module is notified that a drag crossing has occurred.

[0280] 111. The Windows window setting module generates a window crossing instruction corresponding to the window 20a according to the window data, and sends the window crossing instruction of the window 20a to the Windows protocol management module.

[0281] 112. The Windows protocol management module packages the window crossing indication data according to a predetermined protocol.

[0282] 113. The Windows protocol management module sends the obtained window crossing indication data packet to the Android data transmission module via the Windows data transmission module.

[0283] Understandably, both the Windows and Android protocol management modules are primarily responsible for packaging and unpacking the relevant instructions (commands), window parameters, and confirmation information generated during the window dragging process. In other words, all data to be sent to the peer end must be packaged by the current device's protocol management module. This data is then sent by the current device's data transmission module to the peer end's data transmission module.

[0284] Correspondingly, the data packets from the other end also need to be unpacked by the protocol management module of the current device before they can be sent to the corresponding functional module for subsequent processing.

[0285] The protocols followed by the Windows protocol management module for packet assembly and unpacking, and the Android protocol management module for unpacking and assembly, are pre-agreed upon by Android devices and Windows devices. This application does not limit the specific agreement or pre-agreed protocol.

[0286] As shown in Figure 11, the subsequent processing of dragging window 20a on the Windows device can only proceed after the Windows data transmission module receives the window crossing confirmation packet from the Android data transmission module. The Android data transmission module sends the window crossing confirmation packet to the Windows data transmission module through steps 201 through 207 shown in Figure 10. For the specific implementation details of steps 201 through 207, please refer to Figure 15 and the corresponding embodiment section of Figure 15, and will not be further described here.

[0287] 114. After receiving the window crossing confirmation data packet sent by the Android data transmission module, the Windows data transmission module sends the window crossing confirmation data packet to the Windows protocol management module for depacketization processing.

[0288] 115. The Windows protocol management module decapsulates the window crossing confirmation data packet according to the agreed protocol, and obtains a window crossing confirmation message.

[0289] 116. The Windows protocol management module sends the unpacked window crossing confirmation message to the Windows window setting.

[0290] 117. After receiving the window traversal confirmation message for window 20a from the Android device, the Windows window setting module hides window 20a displayed on the screen of the Windows device.

[0291] Specifically, the window 20a is hidden at the specified coordinates, so that when the window 20a is subsequently dragged from the Android device back to the Windows device, the window 20a will appear from the hidden coordinates.

[0292] Alternatively, after the collaborative connection between the Windows device and the Android device is disconnected, the window 20a may automatically appear from the hidden coordinates. Alternatively, after the collaborative connection between the Windows device and the Android device is disconnected, when the user clicks the icon of the document 2 displayed on the Windows device, the window 20a appears from the hidden coordinates in response to the operation.

[0293] It should be understood that the above description is merely an example listed for a better understanding of the technical solution of this embodiment, and is not intended to be the sole limitation on this embodiment.

[0294] Furthermore, it should be noted that the display process of window 20a on the PC can be divided into synthetic rendering, caching, and display. The synthetic rendering of the image corresponding to window 20a (hereinafter referred to as window 20a) is performed, for example, in the window management and image drawing module shown in FIG8A . After the synthetically rendered window 20a is obtained, it is cached in the cache corresponding to window 20a and then retrieved from the cache by the display driver for display.

[0295] Specifically, for the composite drawing canvas, it is usually larger than the display area of ​​the Windows device screen. As shown in Figure 12, the composite drawing canvas can have a visible area (the display interface of the Windows device screen) and an invisible area.

[0296] In the scenario where window 20a needs to be displayed on the Windows device screen, the display driver takes window 20a out of the cache and then displays window 20a in the visible area of ​​the canvas in FIG12 , that is, the display interface of the Windows device screen.

[0297] In the scenario where window 20a does not need to be displayed on the Windows device screen, such as the scenario of hiding window 20a in this embodiment, window 20a is actually still being synthesized and drawn, but after the display driver takes window 20a out of the cache, it sends it to the invisible area of ​​the canvas in Figure 12.

[0298] It should be understood that the above description is merely an example listed for a better understanding of the technical solution of this embodiment, and is not intended to be the sole limitation on this embodiment.

[0299] In addition, in some possible implementations, in order to provide users with a better user experience, by determining the hidden coordinates, and then determining the proportional relationship between the y-axis coordinates of the hidden coordinates and the height yHeight of the Windows device, a floating window 40a is displayed at a position corresponding to the proportional relationship from the side edge of the Android device, such as the left edge as shown in Figure 3C.

[0300] It should be understood that the above description is merely an example listed for a better understanding of the technical solution of this embodiment, and is not intended to be the sole limitation on this embodiment.

[0301] To better understand the determination of the hidden coordinates of the window 20 a , the following description is made with reference to FIG. 13 , FIG. 14A , and FIG. 14B .

[0302] Referring to FIG. 13 , the determination of the hidden coordinates is mainly implemented by the Windows window setting module, and can be specifically implemented through steps S1 to S5 .

[0303] S1, The Windows window setting module determines the width data xWidth and height data yHeight of window 20a according to the window parameters.

[0304] Specifically, the Windows window setting module determines based on the current handle of window 20a provided by the Windows window management module. Regarding the implementation details of determining the width and height of a window according to the window handle, reference can be made to existing literature and will not be elaborated here.

[0305] S2, The Windows window setting module determines the current coordinates (x0, y0) of cursor 10a according to the mouse movement position information.

[0306] S3, The Windows window determines whether x0 ≤ x_min / x0 ≥ x_max.

[0307] Exemplarily, when taking the top left vertex of the Windows device screen as the origin coordinates (0, 0), x_min is 0. x_max is the coordinate corresponding to the X axis among the coordinates of the bottom right vertex of the screen. For example, for a 1920 * 1080 screen, x_max is 1920.

[0308] Based on this, if x0 ≤ x_min, it indicates that the mouse crosses from the left edge of the Windows device screen; if x0 ≥ x_max, it indicates that the mouse crosses from the right edge of the Windows device screen; if x_min < x0 < x_max, it indicates that the mouse is still within the screen of the Windows device, has not reached the left edge, and has not reached the right edge, that is, no crossing will occur.

[0309] When it is determined that no crossing occurs, the Windows window setting module can wait for the window parameters and mouse movement position information obtained at the next moment, and then re - execute step S3 according to the newly obtained information.

[0310] S4, The Windows window determines that the dragged window 20a crosses from the left edge / right edge of the screen.

[0311] S5, The Windows window determines the hidden coordinates (x, y) of window 20a in the Windows device according to the crossing position and the current coordinates of cursor 10a.

[0312] It can be understood that in this embodiment, both the mouse and keyboard and the window cross from the left edge or the right edge of the Windows device screen. Therefore, the Y - axis coordinate of the hidden coordinates can directly be the Y - axis coordinate of the current cursor 10a, such as y0 determined in step S2. The X - axis coordinate needs to be determined according to the crossing position in combination with the width data xWidth.

[0313] Referring to FIG14A , for example, in a scenario where the screen width and height of a Windows device are 1920*1080 and the width and height of window 20a are 200*100, when the user drags window 20a to the right edge of the Windows device screen using the mouse, if the Windows window setting module determines x0 ≥ 1920 according to the processing flow of FIG13 , it can be determined that window 20a and the mouse will cross the right edge of the Windows device screen. For the scenario shown in FIG14A , the hidden coordinates along the X-axis are x=Xright=x_max=1920, and the hidden coordinates along the Y-axis are y=Yright=y0. That is, the hidden coordinates (x, y) are (1920, y0).

[0314] Referring to FIG14B , for example, in a scenario where the screen width and height of a Windows device are 1920*1080 and the width and height of window 20a are 200*100, when the user drags window 20a to the left edge of the Windows device screen with the mouse, when the Windows window setting module determines x0≤0 according to the processing flow of FIG13 , it can be determined that window 20a and the mouse will cross the left edge of the Windows device screen. For the scenario shown in FIG14B , this embodiment is set to hide window 20a only when the entire window 20a is moved out of the Windows device screen. Therefore, the coordinates of the hidden coordinates in the X-axis direction are x=Xleft=-xWidth, and the coordinates in the Y-axis direction are y=Yleft=y0. That is, the hidden coordinates (x, y) are (-xWidth, y0).

[0315] It should be understood that the above description is merely an example listed for a better understanding of the technical solution of this embodiment, and is not intended to be the sole limitation on this embodiment.

[0316] As can be seen from the above description of the embodiment, when window 20a traverses, the first activated floating window 40a is not displayed directly in the center of the Android device screen, but is displayed partially at the edge of the screen near the Windows device, such as the aforementioned floating window 40a that is one-third the size (as shown in FIG3C ). Therefore, even if traversal occurs, the user can still press the left mouse button to continue moving before releasing the mouse, that is, releasing the left mouse button.

[0317] For example, in some possible situations, the user may drag window 20a back to the Windows device before releasing the mouse. In other possible situations, for a scene that crosses from the right edge, the user may continue to move to the right so that the floating window 40a can be fully displayed on the Android device screen.

[0318] Therefore, when the left mouse button is continuously pressed and moved, the Windows system message module will continue to obtain the mouse movement position information and send the mouse movement position information to the Windows window management module for processing, that is, executing step 118.

[0319] 118. The Windows system message module obtains the mouse movement position information and sends the mouse movement position information to the Windows window management module.

[0320] 119. The Windows window management module determines the window movement position information according to the mouse movement position information, and sends the window movement position information to the Windows window setting module.

[0321] Understandably, since the mouse cursor is located on the window during the window dragging process, the window is moved. Therefore, upon knowing the mouse movement position information, the Windows window management module can directly determine the window movement position information corresponding to window 20a.

[0322] 120. The Windows window setting module generates a window movement instruction according to the window movement position information sent by the Windows window management module, and sends the window movement instruction to the Windows protocol management module.

[0323] 121. The Windows protocol management module packages the window movement instruction data according to the agreed protocol, thereby obtaining a window movement instruction data packet.

[0324] 122. The Windows protocol management module sends the obtained window movement instruction data packet to the Android data transmission module via the Windows data transmission module.

[0325] In this way, the Android window device module can change the position of the floating window 40a according to the window movement position information.

[0326] Since the user can release the mouse at any time after the floating window 40a appears on the Android device screen, the operations of steps 123 to 126 may also occur during the execution of steps 118 to 122.

[0327] Continuing to refer to Figure 11, exemplarily, when the user releases the mouse, the Windows system message module will monitor the left mouse button lifted message and the window stop moving message, and send the left mouse button lifted message and the window stop moving message to the Windows window management module for subsequent processing, that is, executing steps 123 and 124.

[0328] 123. The Windows system message module sends a left mouse button lifted message to the Windows window management module.

[0329] 124. The Windows system message module sends a window stop moving message to the Windows window management module.

[0330] 125. After receiving the left mouse button lifted message and the window movement stopped message sent by the Windows system message module, the Windows window management module generates a window release instruction and sends the window release instruction to the Windows window device setting module.

[0331] 126. After receiving the window release instruction sent by the Windows window management module, the Windows window device setting module sends the window release instruction to the Windows protocol management module for package processing.

[0332] 127. The Windows protocol management module packages the window release instruction data according to the agreed protocol, thereby obtaining a window release instruction data packet.

[0333] 128. The Windows protocol management module sends the obtained window release instruction data packet to the Windows data transmission module, which then sends it outward.

[0334] 129. The Windows data transmission module sends the window release indication data packet provided by the Windows protocol management module to the Android data transmission module, so that other functional modules in the Android device can perform subsequent processing based on the window release indication data packet.

[0335] It can be seen from the description of the above embodiment that the cross-device dragging of the window provided by this application also requires the use of the collaborative function to realize the dragging of the window 20a displayed on the Windows device screen to the Android device screen for display. In addition, it can be seen from the description of the above embodiment that when the user releases the mouse, that is, when the loading icon 40a-5 appears on the floating window 40a, the Windows device and the Android device will automatically turn on the collaborative function without the user's perception, and establish a collaborative connection between the two in an extended mode supported by the collaborative function (hereinafter referred to as an extended connection). Therefore, the Windows window management module will execute the operation of turning on the collaborative function, that is, execute step 130, while generating the release window indication.

[0336] 130. The Windows window management module notifies the Windows collaborative service to start the collaborative function while generating the window release instruction.

[0337] 131. The Windows collaborative service sends an extended mode indication to the Windows protocol management module, so that the Windows collaborative management module packages the extended mode indication.

[0338] 132. The Windows protocol management module packages the data indicated by the extended mode according to the agreed protocol, thereby obtaining an extended mode indication data packet.

[0339] 133. The Windows protocol management module sends the obtained extended mode indication data packet to the Windows data transmission module, which then sends it outward.

[0340] 134. The Windows data transmission module sends the extended mode indication data packet provided by the Windows protocol management module to the Android data transmission module, so that other functional modules in the Android device can perform subsequent processing based on the extended mode indication data packet, thereby completing the extended connection between the two.

[0341] In some embodiments, as shown in FIG10 , the Android device must go through at least steps 222 through 227 shown in FIG10 , from receiving the extended mode indication packet to sending the extended connection success packet to the PC, to establish an extended connection between the Windows device and the Android device. The specific implementation details of steps 222 through 227 can be found in FIG15 and the corresponding embodiment section of FIG15 , and are not further described here.

[0342] 135. After receiving the extended connection success data packet sent by the Android data transmission module, the Windows data transmission module sends the extended connection success data packet to the Windows protocol management module for depacketization processing.

[0343] 136. The Windows protocol management module decapsulates the extended connection success data packet according to the agreed protocol and obtains an extended connection success message fed back by the Android device.

[0344] 137. The Windows protocol management module sends an extension connection success message to the Windows window setting module.

[0345] 138. After confirming that the extended connection between the Windows device and the Android device is successful, the Windows window setting module will generate a collaborative screen that needs to be displayed in the display area of ​​the floating window 50a and send it to the Android device for display.

[0346] The generation of the collaborative screen is similar to that of the window 20a, except that the synthetic rendering process is based on the maximized window 20a and the taskbar 30a, so the generated collaborative screen ultimately includes the aforementioned window 20b and taskbar 30b.

[0347] For example, the composite rendering of the collaborative screen is still completed in the window management and image drawing module shown in Figure 8A. After the composite rendered collaborative screen is obtained, it is cached in the cache corresponding to the collaborative screen. Then, the Windows protocol management module packages it according to the agreed protocol, and the Windows data transmission module sends the resulting collaborative screen package to the Android data transmission module.

[0348] Correspondingly, after receiving the collaborative screen package, the Android data transmission module will forward it to the Android protocol management module for unpacking, thereby obtaining the collaborative screen, and finally displaying the obtained collaborative screen in the display area of ​​the floating window 50a.

[0349] For details about the specific content included in the collaborative screen, please refer to the description of the embodiment corresponding to Figure 3E above, which will not be repeated here.

[0350] After completing the operation of step 138 , the Windows setting module will notify the Windows window management module that the dragging of window 20 a is completed, and then execute step 139 .

[0351] At this point, the window 20a displayed on the Windows device screen is dragged to the Android device screen for display, and the corresponding screen change diagrams are shown in FIG. 2A , FIG. 2B , and FIG. 3A to FIG. 3D .

[0352] This concludes the introduction to the operations performed by the various functional modules in the Windows device during the cross-device dragging process. Parts not described in this embodiment can be understood in conjunction with the operations performed by the various functional modules in the Android device and will not be further elaborated here.

[0353] The specific details of steps 201 to 227 implemented by the Android device shown in Figure 10 are described in detail below in conjunction with Figure 15. It should be noted that Figure 15 only shows part of the text, and the specific details can be found in the corresponding text part.

[0354] Referring to FIG15 , in order to implement the cross-device window dragging method provided by the application, the data transmission module, protocol management module, window setting module, virtual data injection module, window setting module, and collaborative service in the Android device execute steps 201 to 227, specifically including:

[0355] 201. After receiving the window crossing indication data packet sent by the Windows data transmission module, the Android data transmission module sends the window crossing indication data packet to the Android protocol management module for depacketization.

[0356] 202. The Android protocol management module unpacks the window crossing indication data packet according to the agreement, and obtains the window crossing indication.

[0357] 203. The Android protocol management module sends the obtained window crossing instruction to the Android window management module for processing.

[0358] 204 , the Android window management module starts the floating window 40 a according to the window crossing instruction.

[0359] In this way, before the extended connection between the Windows device and the Android device is successfully established, the floating window 40a can be used to replace the floating window 50a, so that the user visually feels that the window 20a on the Windows device has indeed been dragged to the Android device and is just waiting for the content of the window 20a to be loaded.

[0360] 205 , the Android window management module generates a window crossing confirmation message and sends the window crossing confirmation message to the Android protocol management module for packaging.

[0361] 206 , the Android protocol management module packages the window crossing confirmation data according to the agreed protocol, and then obtains a window crossing data confirmation data packet.

[0362] 207 , the Android protocol management module sends the obtained window crossing data confirmation data packet to the Windows data transmission module via the Android data transmission module.

[0363] In this way, after receiving the window traversal data confirmation data packet sent by the Android device, the Windows device can execute step 114 and subsequent steps shown in Figure 11.

[0364] 15 , illustratively, after the Android window management module starts the floating window 40a, it will continue to execute steps 208 to 211. That is, steps 208 to 211 can be performed simultaneously with steps 205 to 208, in no particular order.

[0365] 208. After starting the floating window 40a, the Android window management module determines the display position of the floating window 40a on the Android device screen and sends the determined display position information to the Android window setting module to instruct the Android window setting module to set the display position of the floating window 40a.

[0366] 209 , the Android window setting module sets the display position of the floating window 40 a according to the display position information provided by the Android window setting module, so that the floating window 40 a can be displayed at the position specified by the Android screen.

[0367] It should be understood that if the mouse does not have a corresponding down event, when the mouse crosses to the Android device and the user releases the mouse, a mouse up event will occur, so the Windows device cannot perceive it. Before the mouse crosses, the user's left mouse button press operation (down event) is registered on the Windows device. When the mouse crosses from the Windows device to the Android device, there is no corresponding down event in the Android device. Therefore, in order to ensure that the Windows device can be informed when the mouse is released after crossing to the Android device, the Android window setting module will send the position information of the cursor 10b currently corresponding to the mouse on the Android device to the Android virtual data injection module after setting the display position of the floating window 40a, so that the Android virtual data injection module can inject a down event for the mouse on the Android device. That is, execute steps 210 and 211, so that the Windows device will perceive that the mouse has been released.

[0368] 210. The Android window setting module informs the Android virtual data injection module of the mouse position on the Android device, that is, the position information of the cursor 10b corresponding to the mouse on the Android device.

[0369] 211, the Android virtual data injection module injects a down event at the mouse position.

[0370] From the description of the embodiments shown in Figures 3A to 3E, it can be seen that when the mouse and window 20a cross, when the floating window 40a is displayed on the Android device screen for the first time, the floating window 40a is not displayed completely on the Android device screen. Instead, a floating window 40a of about one-third of its size is displayed on the side edge of the screen close to the Windows device, such as the left edge in the above embodiment. Therefore, in order to still be able to drag the floating window 40a to move after the mouse crosses to the Android device, it is necessary to inject the down event corresponding to the mouse at the position where the mouse cursor 10b is displayed on the status bar 1 of the floating window 40a, and to monitor the dragging of the status bar 1. Regarding the implementation of this operation, please refer to the operations performed by the window setting module and HwFreeFormCaptionView in the embodiment shown in Figure 16A, which will not be repeated here.

[0371] Furthermore, as can be seen from the above description of the operations performed by the functional modules in the Windows device, after receiving the window traversal confirmation packet from the Android device, the Windows device generates a window movement instruction packet as the user continues to press the left mouse button and move the mouse, and sends this window movement instruction packet to the Android device. Upon receiving this window movement instruction packet, the Android device executes steps 212 through 216.

[0372] 212. After receiving the window movement instruction data packet sent by the Windows data transmission module, the Android data transmission module sends the window movement instruction data packet to the Android protocol management module for depacketization.

[0373] 213. The Android protocol management module unpacks the window movement instruction data packet according to the agreement, and then obtains the window movement instruction.

[0374] 214. The Android protocol management module sends the obtained window movement instruction to the Android window management module for processing.

[0375] 215 , the Android window management module extracts the window movement position information from the window movement instruction, and sends the extracted window movement position information to the Android window setting module.

[0376] 216. The Android window setting module moves the position of the floating window 40a displayed on the Android screen according to the window movement position information sent by the Android window management module.

[0377] For example, the floating window 40a may be moved from the position shown in FIG. 3C to the position shown in FIG. 3D according to the window movement position information.

[0378] It is understandable that before the user releases the mouse, as the mouse is pressed and moved, the Windows device will continue to send the corresponding window movement indication data packet to the Android device after each movement, and the Android device will continue to move the display position of the floating window 40a according to the window movement position information obtained by unpacking.

[0379] As can be seen from the above description, after the Android virtual data injection module injects a down event into the mouse, when the user releases the mouse, the Windows device can sense it. Then, after processing by the above functional modules within the Windows device, the Windows data transmission module ultimately sends a release window indication packet and an extended mode indication packet to the Android data transmission module. Referring to Figures 10 and 15, it can be seen that upon receiving the release window indication packet, the Android device executes steps 217 to 221, and upon receiving the extended mode indication packet, it executes steps 222 to 227.

[0380] 217. After receiving the release window indication data packet sent by the Windows data transmission module, the Android data transmission module sends the release window indication data packet to the Android protocol management module for depacketization.

[0381] 218. The Android protocol management module unpacks the release window indication data packet according to the agreement, and then obtains the release window indication.

[0382] 219. The Android protocol management module sends the obtained release window instruction to the Android window management module for processing.

[0383] 220 , the Android window management module sends a window release instruction to the Android window setting module, so that the Android window setting module can release the floating window 40 a .

[0384] 221. The Android window setting module releases the floating window 40a.

[0385] At this time, the control 40a-5 can be displayed in the floating window 40a.

[0386] It should be noted that, since the mouse down event is injected at step 211, when the mouse is released and the Android device receives the window release instruction sent by the Windows device, the mouse up event can be injected to release the floating window 40a.

[0387] 222. After receiving the extended mode indication data packet sent by the Windows data transmission module, the Android data transmission module sends the extended mode indication data packet to the Android protocol management module for depacketization.

[0388] 223. The Android protocol management module unpacks the extended mode indication data packet according to the agreement, and obtains the extended mode indication.

[0389] 224. The Android protocol management module sends the obtained extended mode indication to the Android collaborative service for processing.

[0390] Since the window release instruction and the extended mode instruction usually arrive at the Android device relatively quickly, the Android device is already unpacking and transmitting the extended mode instruction data packet during the process of releasing the floating window 40a. Therefore, after the Android window setting module releases the floating window 40a, the Android collaborative service will quickly establish an extended connection with the Windows device and then generate a successful extended connection message.

[0391] 225, the Android collaborative service establishes an extended connection with the Windows device, and after completing the extended connection, generates an extended connection success message, and sends the extended connection success message to the Android protocol management module for packaging.

[0392] 226. The Android protocol management module packages the data of the successful extension connection according to the agreed protocol, thereby obtaining a successful extension connection data packet.

[0393] 227, the Android protocol management module sends the obtained extended connection success data packet to the Android data transmission module, and then the Android data transmission module sends the extended connection success data packet to the Windows data transmission module.

[0394] At this point, window 20a displayed on the Windows device screen has been dragged and dropped onto the Android device screen. The corresponding screen changes are shown in Figures 2A, 2B, and 3A to 3D. The details of replacing floating window 40a shown in Figure 3D with floating window 50a shown in Figure 3E are described in detail below in conjunction with Figure 16A.

[0395] As can be seen from the above description of the embodiment, the floating window 50a displayed on the Android device screen can be enlarged or reduced by dragging, and its size will also change after the screen is rotated. The implementation of these functions requires the use of some functional modules (methods / functions) of the FWK layer in the Android system, such as ActivityStarter, DisplayContent, MultiWindowsManager, FreeFormCaptionView, MouseManager, etc.

[0396] Among them, ActivityStarter is a functional module used to start Activity components, such as floating window 40a and floating window 50a. Different from Android's native ActivityStarter, the ActivityStarter in this application can also set the collaborative mode corresponding to the floating window 50a. For the sake of distinction, the ActivityStarter implemented in this application will be represented as HwActivityStarterEx. It should be understood that this is only for ease of explanation and does not limit the name of the functional module that implements this function.

[0397] Among them, DisplayContent, i.e., the display content manager, is responsible for the process task of prohibiting the floating window 40a from being resized in this application. This ensures that after the floating window 50a is activated, it can completely cover the floating window 40a without the user notifying it, and will not cause the floating window 50a to be unable to fully cover due to the enlargement of the floating window 40a.

[0398] MultiWindowsManager is a multi-window manager. Unlike the native Android MultiWindowsManager, the MultiWindowsManager in this application ensures that the floating window 50a remains in landscape mode even when the Android device's screen is rotated. For clarity, the MultiWindowsManager implemented in this application will be referred to as HwMultiWindowsManager. This is for ease of explanation only and does not limit the name of the functional module that implements this functionality.

[0399] Among them, FreeFormCaptionView can be understood as the taskbar in free window mode, that is, the functional module responsible for managing the status bar 1 and status bar 2 in the floating window 40a and the floating window 50a. Because in the cross-device drag scheme of the window provided in this application, when the cursor 10b is placed in the status bar 1 area, the user can drag the floating window 40a and the floating window 50a by manipulating the mouse. Therefore, unlike the native FreeFormCaptionView of Android, the FreeFormCaptionView in this application can be used to set up drag monitoring for the taskbar. For the sake of distinction, the FreeFormCaptionView implemented in this application will be represented as HwFreeFormCaptionView later. It should be understood that this is only for the convenience of explanation and does not limit the name of the functional module that implements this function.

[0400] Among them, MouseManager, that is, mouse management, is used to manage mouse events, such as down events, up events, drag events, etc. In this application, it is mainly used to manage the mouse drag monitoring of the status bar 1, as well as the cancellation of the drag monitoring after releasing the mouse and the up event. Since the Android device uses the keyboard and mouse of the Windows device in the keyboard and mouse sharing mode, the keyboard and mouse of the Windows device are also managed. For the sake of distinction, the MouseManager implemented in this application will be represented as HnPcMouseManager in the future. It should be understood that this is only for the convenience of explanation and does not limit the name of the functional module that implements this function.

[0401] For ease of description, Figure 16A shows only the Android window management module and Android window settings module in the collaborative service and keyboard and mouse services, as well as the HwActivityStarterEx, DisplayContent, HwMultiWindowsManager, HwFreeFormCaptionView, and HnPcMouseManager listed above in the FWK layer. Specific functional modules for Windows devices are not shown.

[0402] The window management module in the Android device can be represented as DragWindowManager, and the window setting module can be represented as PcScreenCoverActivity. It should be understood that this is only for the convenience of explanation and does not limit the name of the functional module that implements the function.

[0403] Referring to FIG16A , for example, in this embodiment, the extended connection between the Windows device and the Android device is achieved by triggering the collaborative function based on the keyboard and mouse sharing connection between the Windows device and the Android device. In order to ensure that after the extended connection between the Windows device and the Android device is established, the display area of ​​the floating window 50a can display the screen content of the window 20a recorded and delivered by the Windows device without black borders, when the Windows device and the Android device establish a keyboard and mouse sharing connection, the two agree on the aspect ratio of the area displaying the window 20a in the floating window 50a. In this way, the Windows device can subsequently record and deliver the window 20a according to the agreed aspect ratio, and the Android device can also proportionally scale the display according to the same aspect ratio, so that the display area of ​​the floating window 50a can display the window 20a without black borders, and the interface effect is more like directly dragging the window 20a to the Android device display.

[0404] Regarding the aspect ratio agreement, for example, when a Windows device and an Android device establish a keyboard and mouse sharing connection based on the keyboard and mouse sharing service, the Windows device determines the width and height of its current screen and sends it to the Android device. In this way, the Android device can know the width and height of the window 20a that needs to be maximized in the floating window if window crossing occurs later.

[0405] Referring to FIG. 16B , for example, in a scenario where the aspect ratio of a Windows device is 16:9, the aspect ratio agreed upon with an Android device is also 16:9. However, for different forms of extended screen styles in an Android device, the aspect ratio is the ratio of different areas. For example, for the floating window 50a, the aspect ratio of 16:9 indicates the aspect ratio of the remaining area after removing the status bar 1 and status bar 2 from the floating window 50a, that is, the ratio of the width of the floating window 50a to (the height of the floating window 50a - the height of the status bar 1 - the height of the status bar 2) is 16:9.

[0406] Continuing to refer to FIG. 16B , illustratively, for a scenario where the extended screen is the entire screen of the Android device, the aspect ratio of 16:9 indicates the ratio of the area after removing the sidebar 50 a - 7 and the taskbar 30 b .

[0407] It should be understood that the above description is merely an example listed for a better understanding of the technical solution of this embodiment, and is not intended to be the sole limitation on this embodiment.

[0408] Continuing to refer to FIG16A, illustratively, after the Windows device and the Android device have established a keyboard and mouse sharing connection and agreed upon an aspect ratio, if the user presses the left mouse button and the mouse crosses when dragging the window, the hidden coordinates of the window being dragged across can be determined according to the processing logic of the embodiments shown in FIG10 to FIG13, FIG14A, FIG14B and FIG15. In order to ensure that the window that appears first on the Android device, such as the floating window 40a in the above embodiment, can reflect the effect of parallel crossing with the window 20a in the Windows device. The Y-axis height of the upper edge of the floating window 40a should be the same as the Y-axis coordinate of the hidden coordinate. Take the Y-axis coordinate of the hidden coordinate as y0 as an example, as shown in FIG14A and FIG14B.

[0409] Continuing to refer to FIG. 16A , illustratively, after y0b is obtained by the Android window management module, the Android window management module will set the startup bounds of the floating window 40a according to y0 and the agreed aspect ratio, and set the window mode of the floating window 40a to the floating window mode.

[0410] It should be understood that floating window 40a is a virtual window that replaces floating window 50a on the Android device screen before floating window 50a starts to display. From the above description, it can be seen that it is not an extension of the full screen of floating window 50a, but a floating window mode. Therefore, the window mode of floating window 40 that replaces floating window 50a also needs to be set to floating window mode.

[0411] In addition, since the floating window 40a and the floating window 50a are generally the same in size, the area of ​​the floating window 40a that meets the prescribed aspect ratio is the same as that of the floating window 50a, and the following embodiments are represented by area 1 shown in FIG. 16B.

[0412] Since the heights of status bar 1 and status bar 2 are usually fixed, and their widths are the same as the width of region 1, when setting the bounds of floating window 40a, the bounds of region 1 may be determined first.

[0413] In addition, it should be noted that, in some possible cases, the height of status bar 1 and the height of status bar 2 may be the same.

[0414] Furthermore, since floating window 40a is a virtual window that replaces floating window 50a, in this embodiment, the floating window 40a and the floating window 50a are of the same size. In this case, the heights of status bar 1 and status bar 2 in floating window 40a and status bar 1 and status bar 2 in floating window 50a are the same.

[0415] For example, in some possible implementations, such as when it is agreed that area 1 occupies 60% (or other percentage) of the Android device screen, you can first obtain the width / height of the Android device screen, and then divide the width / height of the Android device screen by 60% to obtain the width / height corresponding to area 1. Then, based on the agreed aspect ratio and the determined width / height of area 1, you can get the height / width corresponding to area 1.

[0416] For example, after determining the width of area 1, the X coordinate of the left edge of area 1 (left) can be used as the X coordinate of the upper left corner vertex of the floating window 40a, and the X coordinate of the right edge of area 1 (right) can be used as the X coordinate of the lower right corner vertex of the floating window 40a.

[0417] In addition, it can be known from the above description that y0 is the Y-axis coordinate of the top edge (top) of the floating window 40a, and therefore the Y-axis coordinate of the top left corner of the floating window 40a can be determined to be y0.

[0418] In addition, when the heights of status bar 1 and status bar 2 are fixed and the height of area 1 is determined, and top=y0, the bottom edge of floating window 40a is y0+height of status bar 1+height of area 1+height of status bar 2.

[0419] In this way, the setting of the bounds (left, top, right, bottom) of the floating window 40a can be completed.

[0420] Exemplarily, the setting of the bounds and window mode of the floating window 40a by the Android window management module in FIG16A in the above manner can be used as a specific implementation form of step 208 in the embodiment shown in FIG15 .

[0421] Continuing to refer to Figure 16A, illustratively, after completing the above two steps of setting (setting the bounds of the floating window 40a and setting the window mode to the floating window mode), the Android window management module will notify the Android window setting module to pull up the Activity of the floating window 40a, that is, create the Activity of the floating window 40a.

[0422] Understandably, when the Android window management module notifies the Android window configuration module to launch the floating window 40a's activity, it notifies the Android window configuration module of the parameters set in the above two steps. Thus, when the Android window configuration module calls the StartActivity method and notifies HwActivityStarterEx to draw the floating window 40a's activity, it notifies HwActivityStarterEx of the floating window 40a's bounds and floating window mode as input parameters.

[0423] Continuing to refer to Figure 16A, illustratively, HwActivityStarterEx will draw the Activity of the floating window 40a according to the parameters passed in by the Android window setting module, and notify HwFreeFormCaptionView to start the Caption in the floating window 40a, specifically to set a drag listener for the status bar 1 mentioned above, so that when the cursor 10b falls on the status bar 1, the user can control the mouse to drag the floating window 40a.

[0424] 16A , illustratively, after HwFreeFormCaptionView sets the drag monitoring for status bar 1, it notifies HnPcMouseManager. In this way, HnPcMouseManager can manage the drag monitoring of status bar 1, such as monitoring whether there is an event of dragging status bar 1.

[0425] In addition, it should be noted that HnPcMouseManager will cancel the drag monitoring of status bar 1 after the mouse is released and an up event occurs.

[0426] 16A , illustratively, after HwActivityStarterEx draws the Activity of the floating window 40a, that is, when the Android device screen displays the floating window 40a, DisplayContent will also set the floating window 40a, specifically, to prohibit the floating window 40a from being resized.

[0427] Continuing to refer to FIG16A , illustratively, the Android window setting module will also avoid the hot zone of the status bar 1 of the floating window 40a when determining the floating window 40a, such as the area where the controls 40a-1 to 40a-4 shown in FIG3D are located, and set the initial position of the mouse, that is, the position of the cursor 10b after the mouse passes through.

[0428] Understandably, since the mouse has just passed over and floating window 40a has just been displayed, as shown in FIG3C , the user may continue to drag floating window 40a to the right, for example, until it reaches the position shown in FIG3D , before releasing the mouse. Therefore, the initial position set by the Android window management module for cursor 10b must specifically fall within the area of ​​status bar 1 and avoid the various functional controls displayed on status bar 1.

[0429] In addition, it should be noted that in some possible implementation scenarios, the area on the side edge of the Android device screen is usually set as a hot zone, that is, when the mouse is operated on the side edge, such as sliding, other windows or taskbars will be pulled out. Therefore, the initial position set by the Android window management module for the cursor 10b also needs to avoid the hot zone on the side edge of the Android device screen. Since this case is to display the floating window 40a through the left edge of the Android device screen. Therefore, the initial position set by the Android window management module for the cursor 10b needs to avoid the hot zone on the left edge of the Android device screen.

[0430] For example, when the top of the floating window 40a is known and the height of the status bar 1 is known, in one possible implementation method, the X-axis coordinate of the initial mouse position can be between the left and right determined above, and avoid the coordinates of controls 40a-1 to 40a-4, represented by x1, and the corresponding y1 can be (top + status bar 1 height / 2).

[0431] It should be understood that the above description is merely an example listed for a better understanding of the technical solution of this embodiment, and is not intended to be the sole limitation on this embodiment.

[0432] For example, the setting of the initial position of the mouse (i.e., displaying the cursor 10b in the status bar 1) and the injection of the corresponding down event into the mouse by the Android window device module in FIG16A , as well as the setting and cancellation of the drag monitoring of the status bar 1 by HwFreeFormCaptionView and HnPcMouseManager in the above manner, can serve as a specific implementation form of step 211 in the embodiment shown in FIG15 .

[0433] Continuing to refer to FIG16A , illustratively, after setting the initial position of the mouse, the Android window setting module will inject a down event into the mouse. For details, please refer to steps 209 to 211 in FIG15 , which will not be repeated here.

[0434] Continuing to refer to FIG. 16A , illustratively, after completing the drawing and setting of the floating window 40 a , the Android window setting module will call onResume() to inform the Android window management module that the floating window 40 a has been started normally and is available.

[0435] Continuing to refer to Figure 16A, for example, if after completing the above operations, the Android window management module receives mouse movement information sent by the Windows device, it can cooperate with other functional modules to perform the operations of steps 212 to 216 shown in Figure 15 to realize the movement of the floating window 40a. The specific implementation details are not repeated here.

[0436] Continuing with Figure 16A , for example, if the user releases the mouse, i.e., the Android device detects the mouse up event, HnPcMouseManager will cancel the drag monitoring of status bar 1. Furthermore, as shown in Figures 10 and 11 , after the user releases the mouse, the Windows device sends a window release indication to the Android device and simultaneously initiates the extended mode of the collaborative function, establishing an extended connection with the Android device.

[0437] Continuing to refer to FIG. 16A , illustratively, during the process of establishing an extended connection between the Android device and the Windows device, when HwActivityStarterEx draws the Activity of the floating window 50 a , it first sets the bounds of the floating window 50 a .

[0438] From the above description, it can be seen that the floating window 50a needs to cover the floating window 40a. Therefore, when setting the bounds of the floating window 50a, directly find the bounds of the currently displayed floating window 40a, and then set the bounds of the floating window 50a according to the bounds of the floating window 40a.

[0439] For example, in one possible case, the bounds of the floating window 50a can be the same as the bounds of the floating window 40a. In this way, the floating window 40a can be seamlessly covered, and the user will not feel that the floating window 40a has been replaced.

[0440] In addition, when drawing the Activity of the floating window 50 a , HwActivityStarterEx also needs to set a window module for the floating window 50 a .

[0441] Specifically, if there is a floating window 40a in the current interface of the Android device, the window mode of the floating window 50as is also set to the floating window mode. If there is no floating window 40a, the collaborative mode needs to be changed to the mirror mode, that is, the window mode of the floating window 50a is set to the full screen mode.

[0442] Continuing to refer to FIG. 16A , illustratively, after completing the above settings and drawing the Activity of the floating window 50a according to the above requirements, that is, after starting the floating window 50a, HwActivityStarterEx can call finish() to close the floating window 40a.

[0443] Regarding the process of releasing the mouse and establishing an extended connection, please refer to steps 125 to 137 shown in Figure 11 and steps 217 to 227 in Figure 15. The specific implementation details are not repeated here.

[0444] Continuing with FIG16A , for example, after floating window 50a is displayed, if the Android device screen is rotated, HwMultiWindowsManager will operate to ensure that the floating window 50a is displayed in landscape mode, regardless of whether the Android device screen is in landscape or portrait mode. Regarding the changes in the position and size of floating window 50a when the Android device screen is rotated, please refer to FIG7 .

[0445] Understandably, if the Android device screen rotates while displaying the floating window 40a, HwMultiWindowsManager can also ensure that the display effect of the floating window 40a in the horizontal state can be guaranteed regardless of whether the Android device screen is in the horizontal state or the vertical state.

[0446] This concludes the introduction to the operations performed by the various functional modules in the Android device during the process of dragging a window across devices. Parts not described in this embodiment can be understood in conjunction with the operations performed by the various functional modules in the Windows device and will not be further elaborated here.

[0447] Therefore, when a keyboard and mouse sharing connection is established between a Windows device and an Android device, that is, in a keyboard and mouse sharing scenario, but the collaborative function is not started, when a user uses a Windows device, such as the PC mentioned in the above embodiment to open multiple applications, and needs to use these applications at the same time, such as viewing the contents in the windows corresponding to these applications, based on the cross-device dragging method of windows provided in this application, the window corresponding to the application / document on the PC can be dragged and monitored, and combined with the mouse crossing control in keyboard and mouse sharing, when the mouse crosses, the position of the window moving out of the PC screen is calculated, and the information related to the window is transmitted to the Android device, such as the tablet mentioned in the above embodiment. When the window is released on the tablet (the user releases the left mouse button), the PC and tablet are triggered to connect collaboratively, and the tablet is set as an extended screen of the PC, and then the window hidden on the PC, such as the window of document 2 mentioned in the above embodiment, is moved to the extended screen, that is, the tablet for display, thereby realizing a dual-screen office experience.

[0448] Based on the cross-device window dragging method provided in this application, during the window dragging process, the screen changes of the Windows device and the Android device may be as shown in (1) to (5) in Figure 17.

[0449] In addition, it should be noted that in another possible implementation, the user may sequentially drag multiple windows displayed on the Windows device to the Android device for display. In this implementation scenario, when the floating window 50a described in the above embodiment is not displayed on the Android device, when the user first drags a window on the Windows device, such as the window 20a corresponding to the document 2 in the above embodiment, the process flow of the above embodiment can be followed. During the first dragging of a window from the Windows device to the Android device, the screen changes of the Windows device and the Android device may be as shown in (1) to (5) in FIG. 17 .

[0450] For example, after the window 20a is dragged and displayed in the floating window 50a of the Android device, if the user opens the document 1 in the Windows device, the window corresponding to the document 1 is the window 70a shown in (1) of FIG18 .

[0451] Referring to (1) and (2) in FIG. 18 , for example, when the user operates the mouse 10a to drag the window 70a to move within the screen of the Windows device and gradually drags it to the right edge of the screen of the Windows device, the cursor 10a will cross. Since the collaborative connection between the Windows device and the Android device is not disconnected, the floating window 50a, which is the extended screen of the Windows device, is always displayed on the screen of the Android device. Therefore, when the cursor 10a and the window 70a cross, there is no need to restart the floating window 40a, nor is there a need to re-establish the collaborative connection and the floating window 50a. Therefore, the part of the window 70a that moves out of the Windows device (hereinafter referred to as the first part) and the cursor 10a will directly cross into the floating window 50a, and the part of the window 70a that does not move out of the screen of the Windows device (hereinafter referred to as the second part) will be displayed on the Windows device, as shown in (3) in FIG. 18 . For ease of distinction, this embodiment represents the window 70a that crosses into the floating window 50a as window 70b.

[0452] For example, when window 70a and cursor 10a directly cross to floating window 50a, while cursor 10a drags window 70b, window 70b will continue to follow cursor 10a. Thus, the first portion of window 70a displayed in floating window 50a will become larger and larger, while the second portion of window 70a displayed in the Windows device will become smaller and smaller, until window 70a is completely dragged and displayed in floating window 50a.

[0453] In addition, it should be noted that for the window dragged to the floating window 50a after window 20a, the window style displayed in the restored state (small window) on the Windows device before the window crosses will be maintained, and will be covered on window 20b after the cross, that is, the window that crosses after window 20a, such as window 70b corresponding to window 70a, will block a part of window 20b, as shown in (4) in Figure 18.

[0454] Furthermore, it should be noted that, regardless of whether window 20b or window 70b is dragged and displayed in floating window 50a, the user can edit the content of the window displayed in floating window 50a using the keyboard and mouse in a keyboard and mouse sharing scenario. Furthermore, edits to the content of these windows in floating window 50a will be synchronized with the corresponding documents on the Windows device. For example, if a user adds text to window 20a and saves the edited window 20a, the text will be added to document 2 on the Windows device.

[0455] It should be understood that the above description is merely an example listed for a better understanding of the technical solution of this embodiment, and is not intended to be the sole limitation on this embodiment.

[0456] In addition, it is understandable that, in order to implement the above functions, the electronic device includes hardware and / or software modules corresponding to the execution of each function. In combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to be beyond the scope of this application.

[0457] In addition, it should be noted that the cross-device dragging method of the window provided by the above embodiments implemented by an electronic device in an actual application scenario can also be performed by a chip system included in the electronic device, wherein the chip system may include a processor. The chip system can be coupled to a memory so that when the chip system is running, it calls the computer program stored in the memory to implement the steps performed by the above electronic device. The processor in the chip system can be an application processor or a processor other than an application processor.

[0458] In addition, an embodiment of the present application also provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the cross-device dragging method of the window in the above-mentioned embodiment.

[0459] In addition, an embodiment of the present application further provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes the above-mentioned related steps to implement the cross-device dragging method of the window in the above-mentioned embodiment.

[0460] In addition, an embodiment of the present application also provides a chip (which may also be a component or module), which may include one or more processing circuits and one or more transceiver pins; wherein, the transceiver pins and the processing circuit communicate with each other through an internal connection path, and the processing circuit executes the above-mentioned related method steps to implement the cross-device dragging method of the window in the above-mentioned embodiment to control the receiving pin to receive signals, so as to control the sending pin to send signals.

[0461] In addition, embodiments of the present application further provide a system comprising the aforementioned Windows device and an Android device that establishes a mouse sharing connection with the Windows device. The Windows device and the Android device can implement the cross-device window dragging method provided in the aforementioned embodiments to drag a window from the Windows device to the Android device for display.

[0462] In addition, it can be seen from the above description that the electronic device, computer-readable storage medium, computer program product or chip provided in the embodiments of the present application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0463] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for dragging a window across devices, characterized in that: Applied to a system including a first electronic device and a second electronic device, the operating system of the first electronic device is a Windows system, and the operating system of the second electronic device is an Android system; the method includes: The first electronic device displays a first window; The first electronic device drags the first window to move within the screen of the first electronic device in response to a user operating a mouse cursor to drag the first window; When the mouse cursor moves out of the screen edge of the first electronic device, the first electronic device sends a window crossing instruction for the first window to the second electronic device, and hides the first window displayed on the first electronic device; wherein the mouse corresponding to the mouse cursor is connected to the first electronic device or the second electronic device; The second electronic device displays the first floating window on the screen of the second electronic device in response to the window crossing instruction, and the first window is not displayed in the first floating window; In the process of the mouse cursor dragging the first floating window to move within the screen of the second electronic device, the first electronic device sends a first indication to the second electronic device in response to a user releasing the left button of the mouse; In response to the first instruction, the second electronic device releases the first floating window and displays a second floating window on the screen of the second electronic device, wherein the second floating window covers the first floating window; When the second electronic device receives the first collaborative screen sent by the first electronic device, the second electronic device displays the first collaborative screen in the display area of ​​the second floating window, where the first collaborative screen includes all display contents of the first window.

2. The method according to claim 1, characterized in that After displaying the first collaborative screen in the display area of ​​the second floating window, the method further includes: The first electronic device displays a second window; The first electronic device drags the second window to move within the screen of the first electronic device in response to a user operating a mouse cursor to drag the second window; When the mouse cursor moves out of the screen edge of the first electronic device, the first electronic device sends a second collaborative screen for the second window to the second electronic device, the second collaborative screen includes a first part of the second window, and displays the second part of the second window on the screen of the first electronic device; wherein the first part is the part that moves out of the screen edge of the first electronic device, and the second part is the part that does not move out of the screen edge of the first electronic device; When the second electronic device receives the second collaborative screen sent by the first electronic device, the second collaborative screen is displayed in the display area of ​​the second floating window; wherein the window state of the second window in the second collaborative screen is a restored state, and the second collaborative screen covers the first collaborative screen; Wherein, when the window state is the restored state, the title bar of the first window includes a maximize control but does not include a restore control.

3. The method according to claim 1, characterized in that: When the mouse cursor moves out of the screen edge of the first electronic device, the first electronic device sends a window crossing instruction for the first window to the second electronic device, and hides the first window displayed on the first electronic device, including: When the mouse cursor moves out of the screen edge of the first electronic device, the first electronic device sends a window crossing instruction for the first window to the second electronic device; After sending the window crossing instruction to the second electronic device, the first electronic device hides the first window displayed on the first electronic device when receiving a window crossing confirmation message made by the second electronic device in response to the window crossing instruction.

4. The method according to claim 3, characterized in that After the second electronic device displays the first floating window on the screen of the second electronic device in response to the window crossing instruction, the method further includes: The second electronic device generates the window crossing confirmation message for the window crossing indication, and sends the window crossing message to the first electronic device.

5. The method according to claim 1, characterized in that The first indication includes a release window indication and a screen coordination indication; The release window instruction is used to instruct the second electronic device to release the first floating window; The screen cooperation instruction is used to instruct the second electronic device to establish a screen cooperation connection with the first electronic device in which the cooperation mode is an extended mode; Wherein, the second floating window is displayed on the screen of the second electronic device after the screen cooperative connection is successfully established.

6. The method according to claim 1, characterized in that After the second electronic device releases the first floating window in response to the first instruction and displays a second floating window on the screen of the second electronic device, the method further includes: The second electronic device generates a screen coordination connection success message for the screen coordination indication, and sends the screen coordination connection success message to the first electronic device.

7. The method according to claim 6, characterized in that The first collaborative screen also includes a task bar of the first electronic device; The method further comprises: In response to the screen cooperative connection success message, the first electronic device sets the window state of the first window to a maximized state so that the first window displays all display contents included therein; The first electronic device synthesizes and draws the first collaborative screen based on the first window in the maximized state and the task bar of the first electronic device; and sends the obtained first collaborative screen to the second electronic device.

8. The method according to claim 7, characterized in that When the window state is the maximized state, the title bar of the first window includes a restore control but does not include a maximize control.

9. The method according to any one of claims 1 to 8, characterized in that: Hiding, by the first electronic device, the first window displayed on the first electronic device includes: The first electronic device determines the width and height of the first window according to the handle of the first window; The first electronic device determines the current X coordinate and Y coordinate of the mouse cursor corresponding to the mouse according to the movement information of the mouse; The first electronic device determines, according to the current X-coordinate and Y-coordinate of the mouse cursor, the width of the first window, and the X-coordinate interval corresponding to the screen of the first electronic device, the hidden coordinates of hiding the first window; wherein the X-coordinate interval includes the X-axis coordinate of the left edge of the screen of the first electronic device and the X-axis coordinate of the right edge of the screen of the first electronic device, and the X-axis coordinate of the left edge of the screen is smaller than the X-axis coordinate of the right edge of the screen; The first electronic device hides the first window at the location where the hiding coordinates are located.

10. The method according to claim 9, characterized in that The first electronic device determines, according to the current X coordinate and Y coordinate of the mouse cursor, the width of the first window, and the X coordinate interval corresponding to the screen of the first electronic device, the hiding coordinates for hiding the first window, including: When the X-coordinate of the mouse cursor is not greater than the X-axis coordinate of the left edge of the screen, the first electronic device determines a difference between the X-axis coordinate of the left edge of the screen and the width of the first window; The first electronic device uses the difference as the X coordinate of the hidden coordinate and uses the Y coordinate of the mouse cursor as the Y coordinate of the hidden coordinate.

11. The method according to claim 9, characterized in that The first electronic device determines, according to the current X coordinate and Y coordinate of the mouse cursor, the width of the first window, and the X coordinate interval corresponding to the screen of the first electronic device, the hiding coordinates for hiding the first window, including: When the X coordinate of the mouse cursor is not less than the X axis coordinate of the right edge of the screen, the first electronic device uses the X axis coordinate of the right edge of the screen as the X coordinate of the hidden coordinate and uses the Y coordinate of the mouse cursor as the Y coordinate of the hidden coordinate.

12. The method according to claim 9, characterized in that After the first electronic device hides the first window at the location where the hiding coordinates are located, the method further includes: In response to a drag operation of traversing the first window from the second electronic device back to the first electronic device, the first window is displayed from the hidden coordinates.

13. The method according to any one of claims 1 to 12, characterized in that: The first electronic device displays a first window, including: The first electronic device displays a first window in a maximized state; wherein, when the window state is the maximized state, a title bar of the first window includes a restore control but does not include a maximize control.

14. The method according to claim 13, characterized in that The first electronic device, in response to a user operating a mouse cursor to drag the first window, drags the first window to move within a screen of the first electronic device, including: The first electronic device switches the window state of the first window from the maximized state to the restored state in response to a user operating a mouse cursor to drag the first window; wherein, when the window state is the restored state, the title bar of the first window includes a maximize control but does not include a restore control; The first electronic device drags the first window whose window state is the restored state to move within the screen of the first electronic device.

15. The method according to any one of claims 1 to 12, characterized in that: The first electronic device displays a first window, including: The first electronic device displays a first window whose window state is a restored state; wherein, when the window state is the restored state, a title bar of the first window includes a maximize control but does not include a restore control.

16. The method according to claim 15, characterized in that The first electronic device, in response to a user operating a mouse cursor to drag the first window, drags the first window to move within a screen of the first electronic device, including: In response to the user operating the mouse cursor to drag the first window, the first electronic device drags the first window whose window state is the restored state to move within the screen of the first electronic device.

17. The method according to any one of claims 1 to 16, characterized in that: In the process of dragging the first window to move within the screen of the first electronic device, the method further includes: In the process of dragging the first window to move within the screen of the first electronic device, the first electronic device determines, based on the movement information of the mouse, whether a mouse cursor corresponding to the mouse moves out of the screen of the first electronic device from a left edge of the screen of the first electronic device or a right edge of the screen; When it is determined that the mouse cursor corresponding to the mouse moves out of the screen of the first electronic device from the left edge of the screen of the first electronic device or the right edge of the screen, the first electronic device determines that the mouse cursor moves out of the edge of the screen of the first electronic device.

18. The method according to any one of claims 1 to 17, characterized in that The first electronic device, in response to a user operating a mouse cursor to drag the first window, drags the first window to move within a screen of the first electronic device, including: When the first electronic device monitors the window move message of the first window and the left button of the mouse is pressed, the first electronic device determines that the user operates the mouse cursor to perform the drag operation on the first window; In response to the drag operation, the first electronic device drags the first window to move within the screen of the first electronic device.

19. The method according to any one of claims 1 to 18, characterized in that The first electronic device sends a first indication to the second electronic device in response to a user releasing the left button of the mouse, including: When the first electronic device monitors a message of releasing the left button of the mouse and a message of stopping the movement of the first window, it determines that the user has performed the release operation on the left button of the mouse; The first electronic device sends a first indication to the second electronic device in response to the release operation.

20. The method according to any one of claims 1 to 19, characterized in that The second electronic device includes a screen computer and a foldable screen mobile phone.

21. A method for dragging a window across devices, characterized in that: Applied to a system including a first electronic device and a second electronic device, the operating system of the first electronic device is a Windows system, and the operating system of the second electronic device is an Android system; the method includes: The first electronic device displays a first window; The first electronic device drags the first window to move within the screen of the first electronic device in response to a user operating a mouse cursor to drag the first window; When the mouse cursor moves out of the screen edge of the first electronic device, the first electronic device sends a window crossing instruction for the first window to the second electronic device, and hides the first window displayed on the first electronic device; wherein the mouse corresponding to the mouse cursor is connected to the first electronic device or the second electronic device; The second electronic device displays a first floating window on a screen of the second electronic device in response to the window crossing indication, wherein the first floating window includes a first upper status bar, a first lower status bar, and a first display area, and the first window is not displayed in the first display area; The second electronic device displays the mouse cursor in the first upper status bar and registers a mouse left button press event for the mouse; In the process of the mouse cursor dragging the first floating window to move within the screen of the second electronic device, the first electronic device sends a first indication to the second electronic device in response to a user releasing the left button of the mouse; In response to the first indication, the second electronic device registers a mouse left button lift event for the mouse to release the first floating window and display a second floating window, wherein the second floating window covers the first floating window, and the second floating window includes a second upper status bar, a second lower status bar, and a second display area; When the second electronic device receives the first collaborative screen sent by the first electronic device, the second electronic device displays the first collaborative screen in the second display area, where the first collaborative screen includes all display contents of the first window.

22. The method according to claim 21, characterized in that When the mouse cursor moves out of the screen edge of the first electronic device, the first electronic device sends a window crossing instruction for the first window to the second electronic device, and hides the first window displayed on the first electronic device, the method further includes: The second electronic device establishes a keyboard and mouse sharing connection with the first electronic device, and in the process of establishing the keyboard and mouse sharing connection, agrees on an aspect ratio of the first display area; The aspect ratio of the first display area is the same as the aspect ratio of the screen of the first electronic device.

23. The method according to claim 22, characterized in that The heights of the first upper status bar and the first lower status bar are fixed; The second electronic device displays a first floating window on a screen of the second electronic device in response to the window crossing instruction, including: The second electronic device creates a first Activity in response to the window crossing indication according to the aspect ratio, the screen size of the second electronic device, the height of the first upper status bar, the height of the first lower status bar and a preset first window proportion; and sets the window mode of the first Activity to a floating window mode to obtain the first floating window.

24. The method according to claim 23, characterized in that The second electronic device creates a first Activity in response to the window crossing indication according to the aspect ratio, the screen size of the second electronic device, the height of the first upper status bar, the height of the first lower status bar, and a preset first window proportion, including: The second electronic device determines, according to the aspect ratio, the screen size of the second electronic device and a preset first window proportion, a width of the first display area and a height of the first display area; The second electronic device obtains the Y coordinate of the first window crossing carried in the window crossing indication, and uses the Y coordinate of the first window crossing as the starting Y coordinate of the height of the first Activity; The second electronic device determines, according to the starting Y coordinate, the height of the first upper status bar, the height of the first display area, and the height of the first lower status bar, the ending Y coordinate of the height of the first Activity; The second electronic device sets a first bounds attribute of the first Activity according to the width of the first display area, the start Y coordinate, and the end Y coordinate; The second electronic device creates the first Activity according to the first bounds attribute.

25. The method according to claim 24, characterized in that Display the second floating window, including: The second electronic device creates a second Activity according to the first Activity; and sets the window mode of the second Activity to a floating window mode to obtain the second floating window.

26. The method according to claim 25, characterized in that After displaying the second floating window, the method further includes: The second electronic device destroys the first floating window.

27. The method according to claim 25, characterized in that The second upper status bar has the same height as the first upper status bar, and the second lower status bar has the same height as the first lower status bar; The second electronic device creates a second Activity according to the first Activity, including: The second electronic device obtains a first bounds attribute of the first Activity; When the first bounds attribute is acquired, the second electronic device sets a second bounds attribute of the second Activity according to the first bounds attribute; The second electronic device creates the second Activity according to the second bounds attribute.

28. The method according to claim 27, characterized in that The second bounds attribute is the same as the first bounds attribute.

29. The method according to claim 27, characterized in that The method further comprises: When the first bounds attribute is not obtained, the second electronic device determines the width of the second display area and the height of the second display area according to the aspect ratio, the screen size of the second electronic device and the preset first window proportion; The second electronic device obtains the Y coordinate of the first window crossing carried in the window crossing indication, and uses the Y coordinate of the first window crossing as the starting Y coordinate of the height of the second Activity; The second electronic device determines, according to the starting Y coordinate, the height of the second upper status bar, the height of the second display area, and the height of the second lower status bar, the ending Y coordinate of the height of the second Activity; The second electronic device sets the second bounds attribute of the second Activity according to the width of the second display area, the starting Y coordinate, and the ending Y coordinate.

30. The method according to claim 23, characterized in that Create the first Activity, also include: The window attribute of the first Activity is set to disable scaling mode.

31. The method according to any one of claims 21 to 30, characterized in that The second electronic device displays the mouse cursor in the first upper status bar, comprising: The second electronic device selects a coordinate point from the currently displayed first upper status bar as the initial mouse coordinate according to the display position of the first floating window; wherein the initial mouse coordinate avoids the area where the function keys included in the first upper status bar are located and the hot zone of the side edge of the screen of the second electronic device; The second electronic device displays the mouse cursor at the initial coordinates of the mouse.

32. The method according to any one of claims 21 to 31, characterized in that The first indication includes a release window indication and a screen coordination indication; The release window instruction is used to instruct the second electronic device to release the first floating window; The screen cooperation instruction is used to instruct the second electronic device to establish a screen cooperation connection with the first electronic device in which the cooperation mode is an extended mode; Wherein, the second floating window is displayed on the screen of the second electronic device after the screen cooperative connection is successfully established.

33. The method according to claim 32, characterized in that The method further comprises: The second electronic device registers a drag monitor for the first upper status bar; The second electronic device is further configured to cancel the drag monitoring in response to the release window indication.

34. The method according to claim 33, characterized in that Before canceling the drag monitoring, the method further includes: When the drag monitoring is triggered and the window movement position information sent by the first electronic device is received, the second electronic device moves the display position of the first floating window according to the window movement position information.

35. The method according to any one of claims 21 to 34, characterized in that The second floating window is displayed in the second electronic device whose screen state is in a horizontal state; After displaying the first collaborative screen in the second display area, the method further includes: In response to the screen rotation operation, the second electronic device displays a third floating window at a preset position from the upper edge of the screen in the portrait state, wherein the third floating window includes a third upper status bar, a third lower status bar, and a third display area; The second electronic device displays the first collaborative screen in the third display area; and destroys or hides the second Activity; Among them, the heights of the third upper status bar and the third lower status bar are fixed, the aspect ratio of the third display area is the same as the aspect ratio of the first display area agreed with the first electronic device when establishing the keyboard and mouse sharing connection, and the aspect ratio of the first display area is the same as the aspect ratio of the screen of the first electronic device.

36. The method according to claim 35, characterized in that The method further comprises: The second electronic device scales the second floating window in response to a scaling operation on the second floating window; Among them, the aspect ratio of the second display area of ​​the second floating window after being enlarged or reduced is the same as the aspect ratio of the first display area agreed with the first electronic device when establishing the keyboard and mouse sharing connection, and the aspect ratio of the first display area is the same as the aspect ratio of the screen of the first electronic device.

37. The method according to any one of claims 21 to 36, characterized in that After displaying the first collaborative screen in the second display area, the method further includes: The first electronic device displays a second window; The first electronic device drags the second window to move within the screen of the first electronic device in response to a user operating a mouse cursor to drag the second window; When the mouse cursor moves out of the screen edge of the first electronic device, the first electronic device sends a second collaborative screen for the second window to the second electronic device, the second collaborative screen includes a first part of the second window, and displays the second part of the second window on the screen of the first electronic device; wherein the first part is the part that moves out of the screen edge of the first electronic device, and the second part is the part that does not move out of the screen edge of the first electronic device; When the second electronic device receives the second collaborative screen sent by the first electronic device, the second electronic device displays the second collaborative screen in the second display area; wherein the window state of the second window in the second collaborative screen is a restored state, and the second collaborative screen blocks the first collaborative screen; Wherein, when the window state is the restored state, the title bar of the first window includes a maximize control but does not include a restore control.

38. The method according to any one of claims 21 to 37, characterized in that When the mouse cursor moves out of the screen edge of the first electronic device, the first electronic device sends a window crossing instruction for the first window to the second electronic device, and hides the first window displayed on the first electronic device, including: When the mouse cursor moves out of the screen edge of the first electronic device, the first electronic device sends a window crossing instruction for the first window to the second electronic device; After sending the window crossing instruction to the second electronic device, the first electronic device hides the first window displayed on the first electronic device when receiving a window crossing confirmation message made by the second electronic device in response to the window crossing instruction.

39. The method according to claim 38, characterized in that After the second electronic device displays the first floating window on the screen of the second electronic device in response to the window crossing instruction, the method further includes: The second electronic device generates the window crossing confirmation message for the window crossing indication, and sends the window crossing message to the first electronic device.

40. The method according to any one of claims 21 to 39, characterized in that After the second electronic device registers a left mouse button lift event for the mouse in response to the first indication to release the first floating window and display a second floating window, the method further includes: The second electronic device generates a screen coordination connection success message for the screen coordination indication, and sends the screen coordination connection success message to the first electronic device.

41. The method according to claim 40, characterized in that The first collaborative screen also includes a task bar of the first electronic device; The method further comprises: In response to the screen cooperative connection success message, the first electronic device sets the window state of the first window to a maximized state so that the first window displays all display contents included therein; The first electronic device synthesizes and draws the first collaborative screen based on the first window in the maximized state and the task bar of the first electronic device; and sends the obtained first collaborative screen to the second electronic device.

42. The method according to claim 41, characterized in that When the window state is the maximized state, the title bar of the first window includes a restore control but does not include a maximize control.

43. The method according to any one of claims 21 to 42, characterized in that The second electronic device includes a screen computer and a foldable screen mobile phone.

44. A window cross-device dragging system, characterized in that: include: A first electronic device and a second electronic device, a keyboard and mouse sharing connection is established between the first electronic device and the second electronic device, the operating system of the first electronic device is a Windows system, and the operating system of the second electronic device is an Android system; The first electronic device and the second electronic device are used to execute the cross-device dragging method for a window as described in any one of claims 1 to 20, or to execute the cross-device dragging method for a window as described in any one of claims 21 to 43.

45. An electronic device, characterized in that: The electronic device includes: a memory and a processor, the memory and the processor are coupled; the memory stores program instructions, and when the program instructions are executed by the processor, the electronic device executes the cross-device dragging method of the window as described in any one of claims 1 to 20, or executes the cross-device dragging method of the window as described in any one of claims 21 to 43.

46. ​​A computer-readable storage medium, characterized in that It includes a computer program, which, when running on an electronic device, enables the electronic device to execute the cross-device dragging method for a window as described in any one of claims 1 to 20, or execute the cross-device dragging method for a window as described in any one of claims 21 to 43.

Images