Cross-device window dragging method and device and storage medium

Abstract

The invention provides a cross-device window dragging method and device and a storage medium. The method is applied to a first electronic device of a Windows system and a second electronic device of an Android system, and when a mouse moving by dragging a first window displayed in the first electronic device passes through the second electronic device, the first electronic device sends a window passing instruction to the second electronic device; the second electronic equipment can start the first floating window which does not display the first window, so that a user visually perceives that the first window is traversed. After the mouse is released, the first electronic equipment notifies the second electronic equipment user of releasing the pressing on the left key of the mouse, so that the second electronic equipment can release the first floating window and start the second floating window serving as the expansion screen of the first electronic equipment, and the first window can be displayed in the display area of the second floating window; therefore, cross-device dragging of the window is realized in a keyboard and mouse sharing scene.

Description

Technical Field

[0001] The present application relates to the field of electronic devices, and in particular to a method, device and storage medium for dragging a window across devices. Background Art

[0002] The so-called keyboard and mouse sharing means that multiple electronic devices that establish a keyboard and mouse sharing connection can share a set of input / output (I / O) devices such as keyboard and mouse. That is, in the keyboard and mouse sharing scenario, the user can use the same mouse to operate multiple electronic devices.

[0003] However, with the diversification of user needs, only using the same mouse to traverse different electronic devices and then operate different electronic devices can no longer meet the user's needs. Summary of the invention

[0004] In order to solve the above technical problems, the present application provides a method, device and storage medium for dragging windows across devices, aiming to realize cross-device dragging of windows in a keyboard and mouse sharing scenario, especially dragging windows from Windows devices to Android devices, so as to better meet user usage needs and enhance user experience.

[0005] In a first aspect, the present application provides a method for dragging a window across devices, which is applied to a system including a first electronic device and a second electronic device, wherein the operating system of the first electronic device is a Windows system, and the operating system of the second electronic device is an Android system. The method includes: the first electronic device displays a first window; the first electronic device responds to the user's operation of dragging the first window by operating a mouse cursor, and drags the first window to move within the screen of the first electronic device; when the mouse cursor moves out of the screen edge of the first electronic device, the first electronic device sends a window crossing indication for the first window to the second electronic device, and hides the first window displayed on the first electronic device; wherein the mouse corresponding to the mouse cursor is connected to the first electronic device or the second electronic device; the second electronic device responds to the window crossing indication, and displays a first floating window on the screen of the second electronic device, the first floating window including a first upper status bar, a first lower status bar and a first display area, and the first window is not displayed in the first display area; the second electronic device A mouse cursor is displayed in a first upper status bar, and a left-button-down event is registered for the mouse; while the mouse cursor drags the first floating window within the screen of the second electronic device, the first electronic device sends a first indication to the second electronic device in response to the user releasing the left button of the mouse; the second electronic device registers a left-button-up event for the mouse in response to the first indication to release the first floating window and display a second floating window, wherein the second floating window covers the first floating window, and includes a second upper status bar, a second lower status bar, and a second display area; upon receiving the first collaborative screen sent by the first electronic device, the second electronic device displays the first collaborative screen in the second display area, and the first collaborative screen includes all display contents of the first window.

[0006] The first electronic device is, for example, a PC with a Windows system as described in the following embodiments, and the second electronic device is, for example, a tablet with an Android system as described in the following embodiments.

[0007] The first window is, for example, the window 20a mentioned in the following embodiment.

[0008] The first floating window is, for example, the floating window 40a mentioned in the following embodiment. The second floating window is, for example, the floating window 50a mentioned in the following embodiment.

[0009] The first collaborative screen is, for example, the window 20b and the task bar 30b displayed in the display area of ​​the floating window 50a as described in the following embodiment.

[0010] In the keyboard and mouse sharing scenario, various instructions sent by the first electronic device to the second electronic device, and various messages sent by the second electronic device to the first electronic device, are sent based on the keyboard and mouse sharing connection established between the first electronic device and the second electronic device.

[0011] The first collaborative screen, and the second collaborative screen described below, are transmitted based on the extended connection (screen collaborative connection) established between the first electronic device and the second electronic device.

[0012] In this aspect, the first electronic device and the second electronic device establish a keyboard and mouse sharing connection, that is, they are in a keyboard and mouse sharing scenario.

[0013] Among them, the first upper status bar is, for example, the status bar 1 of the floating window 40a mentioned in the following embodiment; the first lower status bar is, for example, the status bar 2 of the floating window 40a mentioned in the following embodiment; the first display area, that is, the area after the first upper status bar and the first lower status bar are removed from the first floating window, is, for example, the area where the control 40a-5 is displayed in the floating window 40a mentioned in the following embodiment.

[0014] Among them, the second upper status bar is, for example, the status bar 1 of the floating window 50a mentioned in the following embodiment; the second lower status bar is, for example, the status bar 2 of the floating window 50a mentioned in the following embodiment; the second display area, that is, the area after the first upper status bar and the first lower status bar are removed from the second floating window, is, for example, the area where the window 20b and the taskbar 30b are displayed in the floating window 50a mentioned in the following embodiment.

[0015] Among them, it is a mouse left button press event registered by the mouse, that is, the down event mentioned in the following embodiment.

[0016] Among them, it is the left mouse button lift event registered by the mouse, that is, the up event mentioned in the following embodiment.

[0017] Regarding the specific implementation process of dragging the first window from the first electronic device to the second electronic device, reference may be made to the description of steps 101 to 139 and steps 201 to 227 in the following embodiments, which will not be repeated here.

[0018] As a result, in the keyboard and mouse sharing scenario, it is possible to drag windows across devices, especially to drag windows from a Windows device to an Android device.

[0019] According to the first aspect, when the mouse cursor moves out of the screen edge of the first electronic device, the first electronic device sends a window crossing indication for the first window to the second electronic device, and hides the first window displayed on the first electronic device. The method also includes: the second electronic device establishes a keyboard and mouse sharing connection with the first electronic device, and in the process of establishing the keyboard and mouse sharing connection, agrees on the aspect ratio of the first display area; wherein the aspect ratio of the first display area is the same as the aspect ratio of the screen of the first electronic device.

[0020] Therefore, in the process of establishing a keyboard and mouse sharing connection between the first electronic device and the second electronic device, the aspect ratio followed by the two in the screen collaboration scenario is predetermined in advance. In this way, in the screen collaboration scenario, the first electronic device can generate a first collaborative screen according to the agreed aspect ratio, and the area used to display the collaborative screen in the second electronic device satisfies the aspect ratio. Therefore, the collaborative screen subsequently displayed in the second display area will not have black edges, so that the user visually feels that the first window is directly dragged to the second electronic device for display, thereby improving the user experience.

[0021] According to the first aspect, or any implementation method of the first aspect above, the heights of the first upper status bar and the first lower task bar are fixed; the second electronic device displays a first floating window on the screen of the second electronic device in response to a window crossing indication, including: the second electronic device creates a first Activity in response to the window crossing indication according to the aspect ratio, the screen size of the second electronic device, the height of the first upper status bar, the height of the first lower task bar, and a preset first window ratio; and the window mode of the first Activity is set to a floating window mode to obtain a first floating window.

[0022] Understandably, in the Android system, each window is composed of an Activity component. Therefore, after drawing the first Activity, its window mode is set to a floating window mode, and the first floating window can be obtained.

[0023] According to the first aspect, or any implementation of the first aspect above, the second electronic device creates a first Activity in response to a window crossing indication according to the aspect ratio, the screen size of the second electronic device, the height of the first upper status bar, the height of the first lower task bar, and the preset first window ratio, including: the second electronic device determines the width of the first display area and the height of the first display area according to the aspect ratio, the screen size of the second electronic device, and the preset first window ratio; the second electronic device obtains the Y coordinate of the first window crossing carried in the window crossing indication, and uses the Y coordinate of the first window crossing as the starting Y coordinate of the height of the first Activity; the second electronic device determines the ending Y coordinate of the height of the first Activity according to the starting Y coordinate, the height of the first upper status bar, the height of the first display area, and the height of the first lower task bar; the second electronic device sets the first bounds attribute of the first Activity according to the width of the first display area, the starting Y coordinate, and the ending Y coordinate; the second electronic device creates the first Activity according to the first bounds attribute.

[0024] The starting Y coordinate of the height of the first Activity is, for example, the top coordinate in the following embodiment, and the ending Y coordinate of the height of the first Activity is, for example, the bottom coordinate in the following embodiment.

[0025] For the setting of the first bounds property of the first Activity, see the following Fig.16A In the illustrated embodiment, the specific implementation details of the operations performed by the window management module are not repeated here.

[0026] According to the first aspect, or any implementation of the first aspect above, displaying a second floating window includes: the second electronic device creates a second Activity based on the first Activity; and setting the window mode of the second Activity to a floating window mode to obtain a second floating window.

[0027] For details on how to create a second Activity based on the first Activity, that is, how to create a second floating window based on the first floating window, see the following Fig.16A In the illustrated embodiment, the specific implementation details of the operations performed by HwActivityStartEx are not repeated here.

[0028] According to the first aspect, or any implementation of the first aspect above, after displaying the second floating window, the method further includes: the second electronic device destroying the first floating window.

[0029] For example, the destruction operation of the first floating window may be, for example, calling finish(). For the calling logic of this interface, please refer to the following description for Fig.16A In the illustrated embodiment, the specific implementation details of the operations performed by HwActivityStartEx are not repeated here.

[0030] As a result, the resource occupation of the second electronic device is reduced.

[0031] According to the first aspect, or any implementation method of the first aspect above, the second upper status bar is the same height as the first upper status bar, and the second lower status bar is the same height as the first lower task bar; the second electronic device creates a second Activity based on the first Activity, including: the second electronic device obtains a first bounds attribute of the first Activity; when obtaining the first bounds attribute, the second electronic device sets a second bounds attribute of the second Activity based on the first bounds attribute; the second electronic device creates a second Activity based on the second bounds attribute.

[0032] For details on how to create a second Activity based on the first Activity, that is, how to create a second floating window based on the first floating window, see the following Fig.16A In the illustrated embodiment, the specific implementation details of the operations performed by HwActivityStartEx are not repeated here.

[0033] According to the first aspect, or any implementation of the first aspect, the second bounds attribute is the same as the first bounds attribute.

[0034] In this way, the size and layout of the second floating window can be guaranteed to be exactly the same as those of the first floating window, so that the first floating window can be completely covered, and the process of replacing the first floating window with the second floating window will be unaware to the user, further improving the user experience.

[0035] According to the first aspect, or any implementation of the first aspect above, the method also includes: when the first bounds attribute is not obtained, the second electronic device determines the width of the second display area and the height of the second display area according to the aspect ratio, the screen size of the second electronic device and the preset first window ratio; the second electronic device obtains the Y coordinate of the first window crossing carried in the window crossing indication, and uses the Y coordinate of the first window crossing as the starting Y coordinate of the height of the second Activity; the second electronic device determines the ending Y coordinate of the height of the second Activity according to the starting Y coordinate, the height of the second upper status bar, the height of the second display area and the height of the second lower status bar; the second electronic device sets the second bounds attribute of the second Activity according to the width of the second display area, the starting Y coordinate and the ending Y coordinate.

[0036] For details on how to create a second floating window, see Fig.16A In the illustrated embodiment, the specific implementation details of the operations performed by HwActivityStartEx are not repeated here.

[0037] According to the first aspect, or any implementation of the first aspect above, creating a first Activity further includes: setting a window attribute of the first Activity to a disabled scaling mode.

[0038] The setting of the above window attributes is, for example, implemented through DisplayContent described in the following embodiment, and the specific implementation details are not repeated here.

[0039] Therefore, by setting the window property of the first Activity to disable the scaling mode, it can be ensured that when the second bounds are set according to the first bounds, the user cannot drag the first floating window. That is, the first bounds will not change, thereby ensuring that the drawn second floating window can completely cover the first floating window without the situation of being unable to cover.

[0040] According to the first aspect, or any one of the implementation methods of the first aspect above, it is characterized in that the second electronic device displays a mouse cursor in the first upper status bar, including: the second electronic device selects a coordinate point from the currently displayed first upper status bar as the initial mouse coordinates according to the display position of the first floating window; wherein the initial mouse coordinates avoid the area where the function buttons included in the first upper status bar are located and the hot area of ​​the side edge of the screen of the second electronic device; the second electronic device displays the mouse cursor at the initial mouse coordinates.

[0041] For details on setting the initial mouse coordinates, see the following Fig.16A In the illustrated embodiment, the specific implementation details of the operations performed by the window setting module are not repeated here.

[0042] According to the first aspect, or any implementation method of the first aspect above, the first indication includes a release window indication and a screen collaboration indication; the release window indication is used to instruct the second electronic device to release the first floating window; the screen collaboration indication is used to instruct the second electronic device to establish a screen collaboration connection with the first electronic device in which the collaboration mode is an extended mode; wherein, the second floating window is displayed on the screen of the second electronic device after the screen collaboration connection is successfully established.

[0043] The screen coordination indication mentioned in this aspect is, for example, the extended mode indication mentioned in the following embodiments.

[0044] Regarding the operations performed by the second electronic device according to the release window indication and the screen coordination indication, please refer to step 129, step 217 to step 221, step 134, and step 222 to step 227 in the following embodiments, which will not be repeated here.

[0045] According to the first aspect, or any implementation of the first aspect above, the method further includes: the second electronic device registers a drag listener for the first upper status bar; wherein the second electronic device is further used to cancel the drag listener in response to a release window indication.

[0046] Registering a drag listener for the first upper status bar is implemented, for example, by HwFreeFormCaptionView described in the following embodiment, and the specific implementation details are not repeated here.

[0047] The operation of canceling the drag monitoring is implemented, for example, by the HnPcMouseManager described in the following embodiment, and the specific implementation details are not repeated here.

[0048] According to the first aspect, or any implementation of the first aspect above, before canceling the drag listening, the method also includes: when the drag listening is triggered and the window movement position information sent by the first electronic device is received, the second electronic device moves the display position of the first floating window according to the window movement position information.

[0049] Regarding the processing operations involved in moving the first floating window, please refer to step 122, step 212 to step 216 in the following embodiment, which will not be repeated here.

[0050] According to the first aspect, or any implementation of the first aspect above, a second floating window is displayed in a second electronic device whose screen state is a horizontal screen state; after the first collaborative screen is displayed in the second display area, the method also includes: the second electronic device responds to the screen rotation operation, and displays a third floating window at a preset position from the upper edge of the screen in the vertical screen state, and the third floating window includes a third upper status bar, a third lower status bar and a third display area; the second electronic device displays the first collaborative screen in the third display area; and destroys or hides the second Activity; wherein the heights of the third upper status bar and the third lower status bar are fixed, the aspect ratio of the third display area is the same as the aspect ratio of the first display area agreed with the first electronic device when establishing a keyboard and mouse sharing connection, and the aspect ratio of the first display area is the same as the aspect ratio of the screen of the first electronic device.

[0051] For example, when the second floating window is the floating window 50a in the following embodiment, the third floating window can be understood as being in a vertical screen state, such as Figure 7 The floating window 50a in the state shown in (2).

[0052] Correspondingly, the third upper status bar, the third lower status bar and the third display area are the status bar 1, the status bar 2 and the area for displaying the window 20b and the task bar 30b in the floating window 50a in the vertical screen state.

[0053] For the switching between the horizontal and vertical screens, and the position and size changes of the floating window 50a during the switching process, please refer to Figure 7 The description of the illustrated embodiment will not be repeated here.

[0054] According to the first aspect, or any implementation of the first aspect above, the method also includes: the second electronic device scales the second floating window in response to a scaling operation on the second floating window; wherein the aspect ratio of the second display area of ​​the enlarged or reduced second floating window is the same as the aspect ratio of the first display area agreed with the first electronic device when establishing a keyboard and mouse sharing connection, and the aspect ratio of the first display area is the same as the aspect ratio of the screen of the first electronic device.

[0055] Therefore, by setting the second floating window during the scaling process, the second display area also meets the agreed aspect ratio, so that no matter how the second floating window is increased, no black border will appear during the display of the first collaborative screen.

[0056] For the zoom operation on the second floating window, please refer to the following Fig. 6A and Figure 6B The description of the illustrated embodiment will not be repeated here.

[0057] According to the first aspect, or any implementation of the first aspect above, after displaying the first collaborative screen in the display area of ​​the second floating window, the method also includes: the first electronic device displays the second window; the first electronic device drags the second window to move within the screen of the first electronic device in response to the user operating the mouse cursor to drag the second window; when the mouse cursor moves out of the screen edge of the first electronic device, the first electronic device sends a second collaborative screen for the second window to the second electronic device, the second collaborative screen includes a first part of the second window, and displays the second part of the second window on the screen of the first electronic device; wherein the first part is a part that moves out of the screen edge of the first electronic device, and the second part is a part that does not move out of the screen edge of the first electronic device; when the second electronic device receives the second collaborative screen sent by the first electronic device, it displays the second collaborative screen in the display area of ​​the second floating window; wherein the window state of the second window in the second collaborative screen is a restored state, and the second collaborative screen blocks the first collaborative screen; wherein, when the window state is a restored state, the title bar of the first window includes a maximize control but does not include a restore control.

[0058] The second window displayed on the first electronic device, for example Fig.18 The window 70a corresponding to the document 1 shown in (1) is shown in FIG. Fig.18 Neutralize (3) Fig.18 Window 70b is shown in (4).

[0059] The generation process of the second collaborative screen also needs to use the synthetic drawing module. Since the first collaborative screen including the task bar of the first electronic device has been displayed in the second floating window, when generating the second collaborative screen, it only needs to be generated according to the second window.

[0060] For the process of dragging the second window in the first electronic device to the second floating window already displayed in the second electronic device, please refer to the following Fig.18 The embodiment part will not be repeated here.

[0061] According to the first aspect, or any implementation of the first aspect above, when the mouse cursor moves out of the screen edge of the first electronic device, the first electronic device sends a window crossing indication for the first window to the second electronic device, and hides the first window displayed on the first electronic device, including: when the mouse cursor moves out of the screen edge of the first electronic device, the first electronic device sends a window crossing indication for the first window to the second electronic device; after the first electronic device sends the window crossing indication to the second electronic device, when the first electronic device receives a window crossing confirmation message made by the second electronic device for the window crossing indication, the first window displayed on the first electronic device is hidden.

[0062] For more information about the mouse cursor, traversal of the first window, and hiding of the first window, see Fig.12 , Fig.13 , Fig.14A , Fig. 14B , as well as steps 110 to 113, step 207, and steps 114 to 117 in the following embodiments, are not repeated here.

[0063] According to the first aspect, or any implementation of the first aspect above, after the second electronic device displays the first floating window on the screen of the second electronic device in response to the window crossing indication, the method also includes: the second electronic device generates a window crossing confirmation message for the window crossing indication, and sends the window crossing message to the first electronic device.

[0064] Regarding the operations before and after the second electronic device sends the window crossing confirmation message, and the specific process of sending the window crossing message to the first electronic device, please refer to step 113 and step 201 to step 207 in the following embodiment, which will not be repeated here.

[0065] According to the first aspect, or any implementation of the first aspect above, the second electronic device registers a left mouse button lift event for the mouse in response to the first indication to release the first floating window and display the second floating window. The method also includes: the second electronic device generates a screen collaborative connection success message for the screen collaborative indication, and sends the screen collaborative connection success message to the first electronic device.

[0066] The screen coordination indication in this aspect is, for example, the extended mode indication in the following embodiments. The screen coordination connection success message is, for example, the extended connection success message in the following embodiments.

[0067] Regarding the specific implementation details of the extended connection success message (screen collaborative connection success message) made by the second electronic device in response to the extended mode indication (screen collaborative indication) after releasing the first floating window, please refer to step 134 and steps 222 to 227 in the following embodiments, which will not be repeated here.

[0068] According to the first aspect, or any implementation of the first aspect above, the first collaborative screen also includes a taskbar of the first electronic device; the method also includes: the first electronic device sets the window state of the first window to a maximized state in response to a screen collaborative connection success message, so that the first window displays all display content included; the first electronic device synthesizes and draws the first collaborative screen based on the first window with the window state in the maximized state and the taskbar of the first electronic device; and sends the obtained first collaborative screen to the second electronic device.

[0069] The task bar of the first electronic device is, for example, the task bar 30a described in the following embodiment and drawings. For specific implementation details of synthesizing and drawing the first collaborative screen, please refer to step 138 in the following embodiment, which will not be described here.

[0070] According to the first aspect, or any implementation of the first aspect above, when the window state is a maximized state, the title bar of the first window includes a restore control but does not include a maximize control.

[0071] Regarding the style of the title bar of the first window in the maximized state, for example Figure 3E The title bar of the window (window 20b) corresponding to document 2 displayed on the Android device.

[0072] According to the first aspect, or any implementation of the first aspect above, the first electronic device hides the first window displayed on the first electronic device, including: the first electronic device determines the width and height of the first window according to the handle of the first window; the first electronic device determines the current X coordinate and Y coordinate of the mouse cursor corresponding to the mouse according to the movement information of the mouse; the first electronic device determines the hidden coordinates of hiding the first window according to the current X coordinate and Y coordinate of the mouse cursor, the width of the first window and the X coordinate interval corresponding to the screen of the first electronic device; wherein the X coordinate interval includes the X-axis coordinate of the left edge of the screen of the first electronic device and the X-axis coordinate of the right edge of the screen of the first electronic device, and the X-axis coordinate of the left edge of the screen is smaller than the X-axis coordinate of the right edge of the screen; the first electronic device hides the first window at the position where the hidden coordinates are located.

[0073] For more information on how to determine hidden coordinates, see Fig.12 The description of the illustrated embodiment will not be repeated here.

[0074] According to the first aspect, or any implementation of the first aspect above, the first electronic device determines the hidden coordinates of hiding the first window according to the current X coordinate and Y coordinate of the mouse cursor, the width of the first window, and the X coordinate interval corresponding to the screen of the first electronic device, including: when the X coordinate of the mouse cursor is not greater than the X-axis coordinate of the left edge of the screen, the first electronic device determines the difference between the X-axis coordinate of the left edge of the screen and the width of the first window; the first electronic device uses the difference as the X coordinate of the hidden coordinate, and uses the Y coordinate of the mouse cursor as the Y coordinate of the hidden coordinate.

[0075] For the hidden coordinates corresponding to when the first window crosses the left edge of the Windows device screen, see Fig. 14B The description of the illustrated embodiment will not be repeated here.

[0076] According to the first aspect, or any implementation of the first aspect above, the first electronic device determines the hidden coordinates of hiding the first window according to the current X coordinate and Y coordinate of the mouse cursor, the width of the first window, and the X coordinate interval corresponding to the screen of the first electronic device, including: when the X coordinate of the mouse cursor is not less than the X-axis coordinate of the right edge of the screen, the first electronic device uses the X-axis coordinate of the right edge of the screen as the X coordinate of the hidden coordinate, and uses the Y coordinate of the mouse cursor as the Y coordinate of the hidden coordinate.

[0077] For the hidden coordinates corresponding to when the first window crosses the right edge of the Windows device screen, see Fig.14A The description of the illustrated embodiment will not be repeated here.

[0078] According to the first aspect, or any implementation of the first aspect above, after the first electronic device hides the first window at the location where the hidden coordinates are located, the method also includes: in response to a drag operation of crossing the first window from the second electronic device back to the first electronic device, displaying the first window from the hidden coordinates.

[0079] According to the first aspect, or any implementation of the first aspect above, the first electronic device displays a first window, including: the first electronic device displays the first window in a maximized state; wherein, when the window state is a maximized state, the title bar of the first window includes a restore control but does not include a maximize control.

[0080] That is, before the first window is dragged, the first window occupies the entire display area of ​​the Windows device screen. In this scenario, the style of the title bar of the first window is as follows: Figure 3E The title bar of the window (window 20b) corresponding to document 2 displayed on the Android device.

[0081] According to the first aspect, or any implementation method of the first aspect above, the first electronic device responds to the user operating the mouse cursor to drag the first window, and drags the first window to move within the screen of the first electronic device, including: the first electronic device responds to the user operating the mouse cursor to drag the first window, and switches the window state of the first window from a maximized state to a restored state; wherein, when the window state is the restored state, the title bar of the first window includes a maximize control but does not include a restore control; the first electronic device drags the first window whose window state is the restored state to move within the screen of the first electronic device.

[0082] That is, for the first window displayed in the maximized state, it is automatically switched to the restored state first, and then dragged and moved.

[0083] According to the first aspect, or any implementation of the first aspect above, a first electronic device displays a first window, including: the first electronic device displays the first window in a restored state; wherein, when the window is in a restored state, a title bar of the first window includes a maximize control but does not include a restore control. That is, the style of the title bar of the first window is as follows: Figure 2A The title bar of the window (window 20a) corresponding to document 2 displayed on the Windows device.

[0084] According to the first aspect, or any implementation of the first aspect above, the first electronic device responds to the user operating the mouse cursor to drag the first window, dragging the first window to move within the screen of the first electronic device, including: the first electronic device responds to the user operating the mouse cursor to drag the first window, dragging the first window in the restored state within the screen of the first electronic device. That is, directly dragging such as Figure 2A A window (window 20a) corresponding to document 2 is displayed in the Windows device.

[0085] According to the first aspect, or any implementation of the first aspect above, during the process of dragging the first window to move within the screen of the first electronic device, the method also includes: during the process of dragging the first window to move within the screen of the first electronic device, the first electronic device determines, based on the movement information of the mouse, whether the mouse cursor corresponding to the mouse moves out of the screen of the first electronic device from the left edge of the screen of the first electronic device, or the right edge of the screen; when determining that the mouse cursor corresponding to the mouse moves out of the screen of the first electronic device from the left edge of the screen of the first electronic device, or the right edge of the screen, the first electronic device determines that the mouse cursor moves out of the screen edge of the first electronic device.

[0086] Regarding the implementation of this aspect, please refer to the specific description of step 109 and step 110 in the following embodiment, which will not be repeated here.

[0087] According to the first aspect, or any implementation method of the first aspect above, the first electronic device responds to the user operating the mouse cursor to drag the first window, and drags the first window to move within the screen of the first electronic device, including: when the first electronic device monitors the window move message of the first window and the left mouse button press message of the left button of the mouse being pressed, it is determined that the user operates the mouse cursor to drag the first window; the first electronic device responds to the drag operation, and drags the first window to move within the screen of the first electronic device.

[0088] Regarding the user operating the mouse cursor to drag the first window, and the specific operations performed by the first electronic device in response to the dragging operation, please refer to the description of steps 101 to 107 in the following embodiment, which will not be repeated here.

[0089] According to the first aspect, or any implementation of the first aspect above, the first electronic device sends a first indication to the second electronic device in response to the user releasing the left mouse button, including: when the first electronic device monitors the release message of the left mouse button and monitors the message that the first window stops moving, it determines that the user has released the left mouse button; the first electronic device sends the first indication to the second electronic device in response to the release operation.

[0090] The first indication may include the release window indication in step 125 and the extended mode indication (screen coordination indication) in step 131 in the following embodiment.

[0091] Regarding the specific implementation details of the user releasing the left mouse button and the first electronic device sending the first indication to the second electronic device in response to the release operation, please refer to the description of steps 123 to 134 in the following embodiment, which will not be repeated here.

[0092] According to the first aspect, or any implementation of the first aspect above, the second electronic device includes a screen computer or a foldable screen mobile phone.

[0093] Therefore, by selecting an electronic device with a relatively large screen as the second electronic device, the second floating window displayed on the second electronic device can be relatively large, and the first window displayed in the second floating window dragged from the first electronic device to the second electronic device can be tiled, making it easier for the user to view the content in the window.

[0094] In a second aspect, the present application provides an electronic device. The electronic device includes: a memory and a processor, the memory and the processor are coupled; the memory stores program instructions, and when the program instructions are executed by the processor, the electronic device executes instructions of the method in the first aspect or any possible implementation of the first aspect.

[0095] The second aspect and any implementation of the second aspect correspond to the first aspect and any implementation of the first aspect respectively. The technical effects corresponding to the second aspect and any implementation of the second aspect can refer to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, which will not be repeated here.

[0096] In a third aspect, the present application provides a computer-readable medium for storing a computer program, wherein the computer program includes instructions for executing the method in the first aspect or any possible implementation of the first aspect.

[0097] The third aspect and any implementation of the third aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the third aspect and any implementation of the third aspect can refer to the technical effects corresponding to the first aspect and any implementation of the first aspect, which will not be repeated here.

[0098] In a fourth aspect, the present application provides a computer program, comprising instructions for executing the method in the first aspect or any possible implementation of the first aspect.

[0099] The fourth aspect and any implementation of the fourth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the fourth aspect and any implementation of the fourth aspect can refer to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, which will not be repeated here.

[0100] In a fifth aspect, the present application provides a chip, the chip comprising a processing circuit and a transceiver pin, wherein the transceiver pin and the processing circuit communicate with each other through an internal connection path, and the processing circuit executes the method in the first aspect or any possible implementation of the first aspect to control the receiving pin to receive a signal and control the sending pin to send a signal.

[0101] The fifth aspect and any implementation of the fifth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the fifth aspect and any implementation of the fifth aspect can refer to the technical effects corresponding to the first aspect and any implementation of the first aspect, which will not be repeated here.

[0102] In a sixth aspect, the present application provides a system, the system comprising a first electronic device and a second electronic device, the operating system of the first electronic device is a Windows system, and the operating system of the second electronic device is an Android system;

[0103] The first electronic device and the second electronic device may execute the method in the first aspect or any possible implementation manner of the first aspect to implement cross-device dragging of the window.

[0104] The sixth aspect and any implementation of the sixth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the sixth aspect and any implementation of the sixth aspect can refer to the technical effects corresponding to the first aspect and any implementation of the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0105] Figure 1A to Figure 1D This is a schematic diagram of a document dragging scenario in a keyboard and mouse sharing scenario;

[0106] Figure 2A to Figure 2C This is a schematic diagram of a window dragging scenario in a keyboard and mouse sharing scenario;

[0107] Figure 3A to Figure 3E This is a schematic diagram showing an exemplary implementation of window dragging in a keyboard and mouse sharing scenario;

[0108] Figure 4A to Figure 4CA schematic diagram of sidebar styles corresponding to the extended mode in different states of the floating window on the Android device after the Windows device and the Android device establish an extended connection;

[0109] Figure 5 Schematic diagram of the operation of the task bar corresponding to the control 50a-4 on the floating window 50a;

[0110] Fig. 6A and Figure 6B is a schematic diagram showing, for example, shrinking and enlarging the floating window 50a;

[0111] Figure 7 Schematic diagram showing the change of the floating window 50a after the Android device is rotated from the horizontal screen state to the vertical screen state;

[0112] Fig. 8A and Figure 8B A schematic diagram of the software architecture of a Windows device is shown as an example;

[0113] Fig.9A and Fig. 9B A schematic diagram of the software architecture of an Android device is shown as an example;

[0114] Fig.10 A timing diagram of the interaction of functional modules involved in implementing the cross-device dragging method of a window provided in an embodiment of the present application on an exemplary Windows device and an Android device;

[0115] Fig.11 For exemplary purposes Fig.10 A timing diagram of the interaction of functional modules involved when a Windows device implements the cross-device dragging method for a window provided in an embodiment of the present application;

[0116] Fig.12 A schematic diagram showing the relationship between a composite drawing canvas and a screen of a Windows device;

[0117] Fig.13 is a flowchart for calculating the hidden position of the window 20a shown as an example;

[0118] Fig.14A and Fig. 14B is a schematic diagram of an exemplary hidden position of the window 20a;

[0119] Fig.15 For example, Fig.10 A timing diagram of the interaction of functional modules involved when an Android device implements the cross-device dragging method of a window provided in an embodiment of the present application;

[0120] Fig.16A Another timing diagram of the interaction of functional modules involved in implementing the cross-device dragging method of a window provided in an embodiment of the present application on an exemplified Android device;

[0121] Fig. 16B is a schematic diagram showing an exemplary method of determining aspect ratios of extended screens of different sizes;

[0122] Fig.17 This is a schematic diagram showing the interface changes of a Windows device and an Android device when a window 20a is dragged to an Android device for the first time, in accordance with an exemplary method for dragging a window across devices provided in an embodiment of the present application, when the collaborative function is not activated;

[0123] Fig.18 The following is a schematic diagram showing the interface changes of a Windows device and an Android device during the process of dragging a window 70a to an Android device when the collaborative function is started and a floating window 50a is displayed, according to an exemplary method of dragging a window across devices provided in an embodiment of the present application. DETAILED DESCRIPTION

[0124] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0125] The term "and / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0126] The terms "first" and "second" in the description and claims of the embodiments of the present application are used to distinguish different objects rather than to describe a specific order of objects. For example, a first target object and a second target object are used to distinguish different target objects rather than to describe a specific order of target objects.

[0127] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0128] In the description of the embodiments of the present application, unless otherwise specified, the meaning of "multiple" refers to two or more than two. For example, multiple processing units refer to two or more processing units; multiple systems refer to two or more systems.

[0129] With the popularity of electronic devices, more and more users own multiple (at least 2) electronic devices. In order to facilitate users to use multiple electronic devices for office and entertainment at the same time, the keyboard and mouse sharing function has been favored by more and more users. Based on the keyboard and mouse sharing function, electronic devices such as mobile phones and tablets that do not have or are not connected to input / output (I / O) devices such as keyboards and mice can share the use of computer keyboards, mice or touchpads.

[0130] Specifically in this application, electronic devices that do not have or are not connected to I / O devices specifically refer to Android devices (electronic devices whose operating system is the Android system), and computers specifically refer to Windows devices (electronic devices whose operating system is the Windows system).

[0131] In other words, Android devices and Windows devices with keyboard and mouse sharing functions can establish a keyboard and mouse connection, so that the mouse of the Windows device can move freely between the Android device and the Windows device. In this way, users can use applications, view and edit files on different electronic devices by controlling the same mouse.

[0132] Regarding the establishment of a keyboard and mouse sharing connection, in some possible implementation methods, for example, the keyboard and mouse sharing connection can be established by placing the Android device that needs to establish a keyboard and mouse sharing connection next to the Windows device, and then moving the mouse pointer of the Windows device to the side edge of the screen of the Windows device and tapping the side edge of the screen of the Windows device twice in a row.

[0133] Exemplary, in some other possible implementations, for example, you can place the Android device that needs to establish a keyboard and mouse sharing connection next to the Windows device, then enter the Computer Manager in the Windows device, and then establish a keyboard and mouse sharing connection between the Windows device and the Android device placed next to it through the keyboard and mouse sharing entrance under the Smart Interconnection directory in the Computer Manager.

[0134] It should be understood that the above description is only an example for better understanding the technical solution of this embodiment, and is not the only limitation to this embodiment. For specific implementation details of establishing a keyboard and mouse sharing connection between an Android device and a Windows device, please refer to the existing keyboard and mouse sharing document, which will not be repeated here.

[0135] In addition, it should be noted that after establishing a keyboard and mouse sharing connection between an Android device and a Windows device, that is, when the Android device and the Windows device are in a keyboard and mouse sharing scenario, the mouse of the Windows device can not only freely shuttle between the Android device and the Windows device, but in some possible implementation scenarios, the user can also control the mouse of the Windows device to realize the drag and drop transfer of pictures and documents in different electronic devices. For example, the user can use the mouse of the Windows device to transfer a document in the Windows device to the Android device in the form of drag and drop.

[0136] In addition, it should be noted that after completing the above-mentioned drag and drop transfer of the document, the document exists not only in the Windows device, but also in the Android device, so that the user can view the document in both the Window device and the Android device.

[0137] To better understand the above-mentioned drag and drop transfer of documents using the mouse in Windows devices, the following Figure 1A to Figure 1D Provide specific instructions.

[0138] See also Figure 1A For example, in this embodiment, the Windows device is a PC, and the Android devices that establish a keyboard and mouse sharing connection with the PC are a tablet and a folding screen mobile phone. Figure 1A As shown, when the user uses the mouse of the PC to operate the document 2 placed on the PC desktop, for example, moves the mouse cursor to the document 2, presses the left button of the mouse, and drags the document 2, in the keyboard and mouse sharing scenario, as the user presses the left button of the mouse to drag the icon of the document 2, the icon of the document 2 moves with the mouse cursor, such as from Figure 1A The position shown is moved to Figure 1B The location is shown.

[0139] For example, if the user continues to press and hold the left mouse button to drag the icon of document 2, when the mouse cursor moves to the side edge of the PC screen, such as Figure 1C The right edge of the screen in the middle. Through the keyboard and mouse sharing connection between the PC and the tablet, Document 2 will be transferred to the tablet, that is, the icon of Document 2 will be displayed on the tablet screen, such as Figure 1D At the same time, the icon of Document 2 is still displayed on the PC screen.

[0140] That is to say, currently, in the keyboard and mouse sharing scenario, users can use the same mouse to drag and drop pictures and documents in different electronic devices. If the user wants to drag an already opened document, he needs to first close the document window through the close control in the title bar of the document window, and then use the above Figures 1A to 1D Only by dragging the document to other electronic devices can the document be dragged in the manner shown. That is, the currently opened window cannot be dragged from a Windows device to an Android device, that is, the window cannot be dragged across devices.

[0141] In view of this, the present application provides a method for dragging windows across devices. Based on the method for dragging windows across devices provided by the present application, it is possible to drag windows across devices in a keyboard and mouse sharing scenario, especially to drag windows from a Windows device to an Android device.

[0142] Regarding the cross-device dragging of the window mentioned in this application, the changes presented on the screens of the Windows device and the Android device are as follows Figure 2A to Figure 2C .

[0143] See also Figure 2A For example, the Windows device is still a PC, and the Android device that establishes a keyboard and mouse sharing connection with the PC is a tablet. Figure 2A In the keyboard and mouse sharing scenario shown, when the user operates the mouse of the PC, opens the document 2 on the PC, and moves the mouse cursor ( Figure 2A Move the mouse cursor (with the arrow style shown in the figure) to the title bar of the window corresponding to Document 2, press the left mouse button and drag toward the right edge of the PC screen, and the window of Document 2 will move to the right edge of the PC screen along with the mouse cursor.

[0144] See also Figure 2B For example, when the window of Document 2 moves to the right edge of the PC screen along with the mouse cursor, if the user continues to press the left mouse button to move the cursor to the right, based on the cross-device dragging method of the window provided by the present application, the window of Document 2 in the PC will be dragged to the screen of the tablet located on the right side of the PC and with which a keyboard and mouse sharing connection is established, as shown in FIG. Figure 2C That is, based on the cross-device dragging method of the window provided by the present application, the window of Document 2 displayed in the PC will be displayed as a floating window on the screen of the tablet, and at this time, the window of Document 2 will no longer be displayed on the screen of the PC.

[0145] Continue to see Figure 2CFor example, after the window of document 2 is transferred from the PC to the tablet, the mouse connected to the PC will also be transferred to the tablet. That is, the mouse cursor will be transferred from the PC to the tablet. In a possible implementation, after the mouse cursor is transferred from the PC to the tablet, the displayed mouse cursor can be as follows: Figure 2C Furthermore, when the crossing is realized for the first time, the mouse cursor may be on the state 1 of the floating window displayed in the tablet.

[0146] About switching the mouse cursor styles at different positions, setting the initial position, and Figure 3A The functions of the status bar 1 and the status bar 2 shown in FIG. 1 are detailed in the following embodiments and will not be described in detail here.

[0147] Thus, when a keyboard and mouse sharing connection is established between a Windows device and an Android device, that is, when the user uses a Windows device, such as the PC mentioned in the above embodiment to open multiple applications, and needs to use these applications at the same time, such as viewing the contents of the windows corresponding to these applications, the cross-device dragging method of the window provided by this application can be performed by dragging and monitoring the window corresponding to the application / document on the PC, and cooperating with the mouse crossing control in the keyboard and mouse sharing. When the mouse crosses, the position of the window moving out of the PC screen is calculated, and the information related to the window is transmitted to the Android device, such as the tablet mentioned in the above embodiment. When the window is released on the tablet (the user releases the left mouse button), a floating window will be started on the tablet as an extended screen of the PC, and the window of document 2 as mentioned in the above embodiment will be displayed in the floating window, and the window of document 2 will no longer be displayed on the PC, thereby realizing the experience of dragging the window on the PC to the tablet for display.

[0148] In addition, it should be noted that the cross-device dragging method of the window provided in this application is to enable Windows devices in the keyboard and mouse sharing scenario to achieve dual-screen or even multi-screen (more than 3) office experience with the help of Android devices. That is, the window dragged by the mouse also needs to be displayed in the form of a window, so the Android device mentioned in this application can be an Android device with a relatively large screen, such as an Android tablet, an Android folding screen mobile phone, etc. In this way, the currently used window can be tiled for easy viewing of the content in the window by the user.

[0149] In addition, it should be noted that in order to enable the tablet to be used as an extended screen of the PC, and then in the keyboard and mouse sharing scenario, the window in the PC can be dragged and dropped to the tablet for display by the mouse. When implementing the cross-device dragging method of the window provided in this application, the screen collaboration function (hereinafter referred to as the collaboration function) is also required.

[0150] Specifically in each embodiment of the present application, the collaborative function includes: through distributed technology, devices of different systems are collaboratively connected, and then the Sink end (receiving end) is used as an extended screen of the Source end (sending end), thereby realizing a multi-screen office experience.

[0151] Based on the collaborative function, users can achieve a multi-screen (at least 2 screens) experience. For example, multiple files (documents, pictures, etc.) on the Source side, such as file A and file B, can be displayed flatly by opening file A on the Source side and file B on the Sink side, making it convenient for users to view them at the same time.

[0152] It should be understood that the above description is merely an example listed for a better understanding of the technical solution of this embodiment, and is not intended to be the sole limitation to this embodiment.

[0153] In addition, it should be noted that for the collaborative function, its collaborative mode can be divided into mirror mode and extended mode. The so-called mirror mode can be understood as the content displayed in the floating window displayed on the tablet is exactly the same as that on the PC screen. The so-called extended mode can be understood as the content not displayed on the PC screen is extended to be displayed in the floating window displayed on the tablet.

[0154] Based on this, in order to achieve the effect of dragging the window corresponding to document 2 on the PC to be displayed on the tablet, this application adopts the extension mode in the collaborative mode.

[0155] In order to better understand the process of dragging a window from a Windows device, to collaboratively connecting an Android device with a Windows device, starting the extended mode of the collaborative function, and then displaying the window dragged from the Windows device on the screen of the Android device, the following detailed description of the changes in the screens of the Windows device and the Android device during this process is provided in conjunction with the accompanying drawings.

[0156] Still taking the Windows device mentioned in the above embodiment as a PC, the window to be dragged as the window of document 2 displayed on the PC, and the Android device to be dragged as a tablet as an example. For ease of explanation, in this embodiment, the arrow-shaped cursor corresponding to the mouse on the PC is called cursor 10a, the dot-shaped cursor corresponding to the tablet after the mouse is moved to the tablet is called cursor 10b, the window corresponding to document 2 on the PC is called window 20a, the window corresponding to document 2 on the tablet is called window 20b, the taskbar displayed at the bottom of the PC screen is called taskbar 30a, the floating window displayed before the collaborative function is started after the mouse is moved to the tablet is called floating window 40a, the floating window (equivalent to the extended screen of the PC) that replaces the floating window 40a after the collaborative function is started is called floating window 50a, the taskbar 30a displayed in the floating window 50a is represented as taskbar 30b, the status bar at the top of the floating window 40a and the floating window 50a are both called status bar 1, and the status bar at the bottom is both called status bar 2.

[0157] It should be understood that the above description is merely an example listed for a better understanding of the technical solution of this embodiment, and is not intended to be the sole limitation to this embodiment.

[0158] In addition, it should be noted that in the keyboard and mouse sharing scenarios described in the embodiments of the present application, the mouse that can be shared by Windows devices and Android devices is Figures 1A to 1D The mouse connected to the PC device is shown as an example. The mouse is no longer shown in the subsequent figures.

[0159] Exemplarily, when the user holds down the left mouse button and drags the window 20a to the right edge of the PC screen (eg Figure 2B If the left mouse button is not released and the window 20a is dragged to the right while the left mouse button is pressed, the cursor 10a will gradually approach the right edge of the PC screen. In order to achieve the effect of window 20a passing from the PC to the tablet, as the cursor 10a gradually approaches the right edge of the PC screen, part of the window 20a will be moved out of the PC screen, that is, only the part before the cursor 10a is displayed on the PC screen, for example Figure 3A As shown. Figure 3A In the figure, the cursor 10a has not moved to the right edge of the PC screen, that is, the mouse has not crossed, so the cursor 10b and the window 20a will not be displayed on the tablet.

[0160] See also Figure 3BFor example, the user presses the left mouse button to continue dragging window 20a. When cursor 10a moves to the right edge of the PC screen, based on the keyboard and mouse sharing connection established between the PC and the tablet, the PC mouse will pass from the PC to the tablet, that is, cursor 10a will pass to the tablet and change to the style of cursor 10b. At the same time, a floating window 40a of about one-third of the size will appear on the tablet, and window 20a on the PC will be hidden, as shown in FIG. Figure 3C shown.

[0161] It should be noted that before the user releases the left mouse button to start the collaborative function and displays the floating window 50a as an extended screen of the PC on the tablet, the user can move the mouse back to the PC at any time, and the dragged window 20a will also return to the PC screen display. Therefore, before starting the collaborative function, in order to visually make the user feel that the window 20a has crossed from the PC to the tablet, the floating window 40a will be displayed first. That is, the floating window 40a is not an extended screen of the PC, but only a floating window drawn and displayed on the tablet. After the floating window 50a as an extended screen of the PC is started, the floating window 50a will cover the floating window 40a, and the floating window 40a will be destroyed.

[0162] Based on this, in order to prevent the user from feeling that the floating window 40a is replaced by the floating window 50a, in some possible cases, the floating window 40a and the floating window 50a can be drawn as the same size, and the status bar 1 and the status bar 2 have the same function of the selection control.

[0163] This embodiment is described by taking the function control set in the status bar 1 as an example, but in actual applications, the function control may also be set in the status bar 2 according to business requirements.

[0164] like Figure 3D and Figure 3E As shown, the control 40a-1 in the floating window 40a has the same function as the control 50a-1 in the floating window 50a, both of which are used to close the floating window currently displayed on the tablet screen. As for the operation of the control 50a-1, since the collaboration function has been turned on at this time, if the control is clicked, the collaboration between the tablet and the PC will be disconnected, that is, the tablet will no longer serve as an extended screen of the PC.

[0165] Continue to see Figure 3D and Figure 3E, the control 40a-2 in the floating window 40a has the same function as the control 50a-2 in the floating window 50a, and is used to maximize the floating window currently displayed on the tablet screen. The control 40a-3 in the floating window 40a has the same function as the control 50a-3 in the floating window 50a, and is used to minimize the floating window currently displayed on the tablet screen. The control 40a-4 in the floating window 40a has the same function as the control 50a-4 in the floating window 50a, and is used to pop up a hidden taskbar at the upper edge of the current floating window.

[0166] Continue to see Figure 3D For example, if the user keeps pressing the left mouse button to drag the floating window 40a, the user will not release the button until the floating window 40a is completely displayed on the screen of the tablet. When the mouse is released, the PC will send an instruction to release the window 40a to the tablet based on the keyboard and mouse sharing connection. After receiving the instruction to release the window 40a, the tablet will release the window 40a. The operation of releasing the window 40a can be understood as the tablet injecting an up event into the mouse. The specific implementation details of this operation can be found in the description of step 221 in the following embodiment, which will not be repeated here.

[0167] When the mouse is released and the floating window 40a is displayed on the tablet, the PC will start the collaborative function and establish a collaborative connection with the tablet in the selected extended mode. After the collaborative connection is established, the tablet will create a floating window 50a, decompress the collaborative screen transmitted by the PC through the collaborative connection, and then display the collaborative screen in the display area of ​​the floating window 50a.

[0168] The display area of ​​the floating window 50a is the area of ​​the floating window 50a minus the status bar 1 and the status bar 2.

[0169] In addition, since the floating window 50a is used as an extended screen of the PC, the task bar 30a of the PC will be displayed in the floating window 50a. Therefore, the collaborative screen needs to include the window 20b and the task bar 30b.

[0170] In addition, in order to give users a better user experience, in one possible implementation, the collaborative screen can be set to occupy the entire display area of ​​the floating window 50a. In this way, the user visually sees that window 20a is dragged to the tablet as a whole. Therefore, when generating the collaborative screen, the synthesis (drawing) rendering module in the PC specifically draws window 20b based on window 20a in maximized mode. That is, window 20b is essentially window 20a in maximized mode.

[0171] For example, for the window 20a that is not displayed maximized in this embodiment, the function controls displayed in the title bar include a maximize control for maximizing the window 20a, such as Figure 2A As shown. For the window 20b drawn based on the window 20a in the maximized mode, since it is essentially the window 20a in the maximized mode. Therefore, in each embodiment of the present application, the function control displayed in the title bar of the window 20b displayed in the floating window 50a dragged to the tablet will not include the maximize control, but will be replaced by the restore control, such as Figure 3C shown.

[0172] In this way, after the floating window 40a is covered with the floating window 50a, the user can directly see the window 20b. Figure 3E shown.

[0173] Continue to see Figure 3D , for example, while waiting for the PC to establish a collaborative connection with the tablet, using the floating window 50a to cover the floating window 40a, the floating window 40a may also display a dynamic loading icon, such as Figure 3D The icon 40a-5 is shown in the figure to prompt the user that the window 20a is currently being loaded.

[0174] It should be understood that the above description is merely an example listed for a better understanding of the technical solution of this embodiment, and is not intended to be the sole limitation to this embodiment.

[0175] See also Figure 3E , for example, due to the cross-device dragging solution of the window provided in this embodiment, the entire screen of the tablet is not used as the extended screen of the PC, but a floating window 50a is first displayed on the tablet, and the display area in the floating window 50a is used as the extended screen of the PC. Therefore, in order not to block the content in the window 20a in the floating window 50a, the sidebar that will be displayed at the edge of the extended screen (usually on the left / right edge of the screen for tablets and on the upper / lower edge of the screen for folding screen phones) in the extended mode of the collaborative function will be hidden. That is, the sidebar is not displayed in the floating window 50a.

[0176] Exemplarily, in some possible implementations, a hot zone can be set on the right edge of the floating window 50a. When the user moves the cursor 10b to this area, presses the left mouse button to slide to the left, or clicks / double-clicks the left mouse button, and then the hidden sidebar is displayed on the right edge of the floating window 50a.

[0177] For example, in some other possible implementations, an operable control may also be provided on the right edge of the floating window 50a, such as Figure 3E When the user moves the cursor 10b to the control 50a-5 and clicks the left button of the mouse, the hidden sidebar can be displayed on the right edge of the floating window 50a.

[0178] It should be understood that the above description is only an example for better understanding the technical solution of this embodiment, and is not the only limitation of this embodiment. Figure 3E The form of control 50a-5 is shown for illustration.

[0179] From the above definition of cursor style, it can be seen that when the cursor acts on the PC, its style is the style of cursor 10a, and when the cursor acts on the tablet, its style is the style of cursor 10b. Since the floating window 50a is used as an extended screen of the PC, when the cursor moves to the display area inside the floating window 50a, that is, Figure 4A When the area of ​​the window 20b and the task bar 30b is displayed in the display, it is equivalent to acting on the PC, so the cursor will switch from the style of the cursor 10b to the style of the cursor 10a.

[0180] See also Figure 4A In (1), when the cursor 10a moves to the control 50a-5 and the user clicks the left mouse button, the control 50a-5 disappears and is replaced by the following Figure 4A Sidebar 50a-6 shown in (2).

[0181] Understandably, the sidebar 50a-6 will display functional options related to the extended mode, subject to the following conditions: Figure 4A The size of the area (2) is not shown in the sidebar 50a-6. In order to better understand the functional options included in the sidebar, the following Figure 4B Provide explanation.

[0182] See also Figure 4B For example, as an optional situation, the sidebar 50a-6 may include a function option for returning to the desktop, a function option for saving the content in the extended screen, a function option for undoing the last operation, a function option for moving the current window to the opposite end display, a function option for switching the collaborative mode, a function option for port collaboration, etc., which are not listed one by one here, and this embodiment does not limit this.

[0183] Understandably, in one possible implementation, the sidebar 50a-6 can be set to cancel display after any function option in the sidebar 50a-6 is selected. In another possible implementation, the sidebar 50a-6 can also be set to automatically disappear after being displayed for a set period of time, and the control 50a-5 can be redisplayed in the floating window 50.

[0184] It should be understood that the above description is merely an example listed for a better understanding of the technical solution of this embodiment, and is not intended to be the sole limitation to this embodiment.

[0185] In addition, regarding the function option for switching the collaborative mode, when the current mode is extended, if the function option is selected, the collaborative mode will switch from the extended mode to the mirror mode. Conversely, when the current mode is mirror, if the function option is selected, the collaborative mode will switch from the mirror mode to the extended mode. Due to the difference between the mirror mode and the extended mode, the function options displayed in the sidebar will also change, which can be determined according to the two collaborative modes, and this embodiment does not limit this.

[0186] Continue to see Figure 3E For example, if the user operates the mouse to move the cursor 10b to the control 50a-2 and clicks the left button of the mouse, the floating window 50a will be maximized and displayed on the screen of the tablet, such as Figure 4C In this case, the status bar 1 and status bar 2 of the floating window 50a will not be displayed, but in the maximized mode, the left or right side of the tablet screen (the left side is used as an example in this embodiment) will display the sidebar corresponding to the current collaborative mode, such as Figure 4C Still taking the current collaborative mode as the extended mode as an example, the sidebar 50a-7 may display the same functional options as those in the sidebar 50a-6.

[0187] Since the window 20b in the collaborative screen is drawn based on the maximized window 20a, after the floating window 50a is maximized, the collaborative screen received from the PC will also be enlarged as a whole. Therefore, when the floating window 50a is maximized, the window 20b and the taskbar 30b will still occupy the entire display area of ​​the floating window 50a. Figure 4C shown.

[0188] In addition, the operation of the floating window 50a can also be realized through the function options in the task bar corresponding to the control 50a-4. For example, when the cursor 10b moves to the control 50a-4 and the user clicks the left button of the mouse, a hidden task bar can pop up in the area where the control 50a-4 is located, such as Figure 5 The task bar 50a-8 shown in (1).

[0189] See also Figure 5 In (1), exemplarily, in some possible implementations, the taskbar 50a-8 may include functional controls that are the same as controls 50a-1, 50a-2, and 50a-3, as well as functional options for switching collaborative modes shown in the sidebar 50a-6 or the sidebar 50a-7.

[0190] Exemplarily, when the user clicks on the same functional control as the control 50a-2 in the task bar 50a-8, the floating window 50a will be maximized, that is, the tablet displays the interface content as shown in (2) in FIG. 4 .

[0191] For example, when the user clicks the same function control as control 50a-3 in task bar 50a-8, floating window 50a will be minimized. In a possible implementation, the minimized floating window will be displayed as Figure 5 When the user clicks the capsule 60a, the floating window 50a will be redisplayed in the tablet interface, for example, in the style shown in FIG. 4 (1).

[0192] For example, when the user clicks on the same functional control as control 50a-1 in task bar 50a-8, the tablet will disconnect the collaborative connection with the PC and close the floating window 50a. That is, the tablet will return to the style before the window 20 was dragged to the tablet, such as Figure 3B The flat plate shown.

[0193] It should be noted that when the collaborative connection between the tablet and the PC is disconnected, in some possible implementations, the PC may automatically display the window 20a. In other possible implementations, the user may click a designated position or control to redisplay the window 20a on the PC screen.

[0194] It should be understood that the above description is merely an example listed for a better understanding of the technical solution of this embodiment, and is not intended to be the sole limitation to this embodiment.

[0195] Exemplarily, when the user clicks the function option for switching the collaborative mode in the task bar 50a-8, the collaborative mode will be switched, and as the collaborative mode is switched, the content displayed in the floating window 50a will also change. For example, when switching to the mirror mode, the current desktop of the PC will be displayed in the floating window 50a. When switching back to the extended mode, the window dragged to the floating window 50a will be displayed, such as the above-mentioned window 20a.

[0196] It can be seen from the above description that the taskbar 50a-8 will be displayed after the control 50a-4 is clicked, and the control 40a-4 in the floating window 40a has the same function as the control 50a-4. Therefore, in order to avoid the cursor 10b displayed on the tablet interface clicking on the control 40a-4 when the mouse just crosses from the PC to the tablet, when the cursor 10a crosses from the PC to the tablet, the cursor 10b that appears in the tablet can be displayed in the status bar 1 of the floating window 40a, and is far away from the control 40a-4, such as Figure 3C shown.

[0197] It should be understood that the above description is merely an example listed for a better understanding of the technical solution of this embodiment, and is not intended to be the sole limitation to this embodiment.

[0198] In addition, it should be noted that, in this embodiment, the user can also zoom in or out the floating window 50a displayed on the tablet by controlling the mouse.

[0199] Exemplarily, when the user moves the cursor to the edge of the floating window 50a, that is, removes the display area of ​​the floating window 50a, the cursor style will be switched back to the cursor 10b.

[0200] See also Fig. 6A In (1), for example, when the cursor 10b moves to the lower right corner of the floating window 50a, the user presses the left button of the mouse and drags the floating window 50a in the direction of the arrow, and the floating window 50a is reduced in size. Fig. 6A After the user releases the left mouse button, the floating window 50a will shrink to Fig. 6A The size of (2) and stays in that position.

[0201] See also Figure 6B In (1), for example, when the cursor 10b moves to the lower right corner of the floating window 50a, the user presses the left button of the mouse and drags the floating window 50a in the direction of the arrow, and the floating window 50a is enlarged as a whole. Figure 6B After the user releases the left mouse button, the floating window 50a will be enlarged to Figure 6B The size of (2) and stays in that position.

[0202] In addition, it should be noted that, in order to ensure the experience effect, in the window dragging method provided in this embodiment, when the tablet displaying the floating window 50a is rotated from the horizontal screen state to the vertical screen state, such as rotating from the arrow direction in (1) in 7 to Figure 7 After (2), the floating window 50a will also rotate as a whole, and the size of the floating window 50a will be adjusted according to the horizontal distance after the vertical screen.

[0203] In addition, in some possible implementations, when rotating from a horizontal screen to a vertical screen, or from a vertical screen to a horizontal screen, the floating window 50a will be displayed centered on the left and right sides. However, when the user drags and moves, the floating window 50a will stay at the position where the user drags.

[0204] It should be understood that the above description is merely an example listed for a better understanding of the technical solution of this embodiment, and is not intended to be the sole limitation to this embodiment.

[0205] In order to better understand the cross-device drag method of the window provided by this application, the following Fig. 8A , Figure 8B , Fig.9A and Fig. 9B, respectively describing the software architecture of Windows devices and the software architecture of Android devices for implementing a method for dragging a window across devices.

[0206] It should be noted that in actual applications, whether it is a Windows device or an Android device, its software system (operating system) can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. In addition, for Windows devices and Android devices, they can be divided into user state and kernel state.

[0207] See also Fig. 8A , illustrating a software architecture for Windows devices.

[0208] like Fig. 8A As shown, from the perspective of user state, from bottom to top, it can include NTDLL, service process, system process, environment subsystem, data transmission module, user process, etc. Specifically in this embodiment, the cross-device dragging of windows mainly depends on the keyboard and mouse sharing service and collaborative service in the service process, as well as the data transmission mode implementation. Among them, the keyboard and mouse sharing service (process) is a service process that realizes the above-mentioned keyboard and mouse sharing function; the collaborative service (process) is a service process that realizes the above-mentioned collaborative function, that is, it is responsible for establishing a collaborative connection between Windows devices and Android devices; the data transmission module is responsible for the transmission of signaling and data. In one implementation method, when both Windows devices and Android devices are electronic devices generated by Honor manufacturers, the data transmission module, for example, is MagicLink.

[0209] For the functions of other processes, systems, NTDLL, etc. in user mode, please refer to the introduction of the existing Windows system architecture, which will not be repeated here.

[0210] In addition, it should be noted that in some possible implementations, the user process may include a process corresponding to an application similar to the computer manager mentioned above. Through the process corresponding to the computer manager (such as the service management process), the management of various services and the scheduling of various virtualized services can be achieved. For example, the scheduling of keyboard and mouse sharing services and collaborative services in the service process.

[0211] In addition, it should be noted that, since this embodiment realizes the cross-device dragging of windows in the original keyboard and mouse sharing scenario, the keyboard and mouse sharing service is improved. The improved keyboard and mouse sharing service at least includes: Figure 8B The four functional modules are shown.

[0212] See also Figure 8B, exemplarily, the keyboard and mouse sharing service in Windows devices includes a window management module, a system message module, a window setting module, and a protocol management module.

[0213] Among them, the window management module is responsible for the management of dragging windows, the management of window messages, position identification, etc.; the window setting module is responsible for the position calculation of window traversal, hiding, and display; the system message is responsible for the message of the mouse starting to drag the window or stopping to drag the window; the protocol management is responsible for the packaging and unpacking of data such as window dragging related commands, window parameters, etc.

[0214] It should be noted that in Windows programming, after each Windows application starts executing, the system will create a message queue for the application. This message queue is used to store the information of the window created by the application. For example, when the user presses the mouse on a window, a WM_RBUTTONDOWN message will be generated. The system will automatically put this message into the message queue of the application to which the current window belongs, waiting for the end of the application. Windows will put the generated messages into the message queue one by one, and the application will continuously read messages from the message queue through a message loop and respond. When the mouse moves or stops to drag the window, the system will generate a system message of window movement, that is, the message managed by the above-mentioned system message module.

[0215] Continue to see Fig. 8A , for example, from the perspective of kernel state, from bottom to top, it may include hardware abstraction layer, kernel and device driver layer, window management and image drawing module, and Windows executive body. Specifically, in the technical solution provided in this embodiment, the cross-device dragging of windows not only needs to rely on the keyboard and mouse sharing function, but also needs to rely on the collaborative function. Both of these functions need to rely on WiFi or Bluetooth, so it is necessary to involve functional modules related to WiFi and Bluetooth, such as WiFi driver, Bluetooth driver, etc.

[0216] In addition, for Windows devices that do not have I / O devices, they also need to use USB interfaces, Bluetooth, or WiFi to connect external keyboards, mice, etc. Therefore, in addition to the functional modules related to WiFi and Bluetooth, they also need to involve functional modules related to USB, such as USB drivers.

[0217] For descriptions of the Windows executive body in kernel state, other functional modules involved in the kernel and device driver layer, window management and drawing modules, hardware abstraction layer, etc., please refer to the introduction to the existing Windows system architecture, which will not be repeated here.

[0218] This is the end of the introduction to the software architecture of Windows devices. It is understandable that: Fig. 8A and Figure 8B The layers in the software Windows device and the functional modules contained in each layer shown do not constitute a specific limitation on the Windows device. In some possible implementations, the Windows device may include more or fewer layers than shown in the figure, and each layer may include more or fewer components, which is not limited in this application.

[0219] See also Fig.9A , illustrating a software architecture for Windows devices.

[0220] like Fig.9A As shown in Figure 1, the layered architecture of Android devices divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some implementations, the Android system is divided into five layers, from top to bottom: the application layer (Application, APP layer), the framework layer (Application Framework, FWK layer), the Android runtime (Android runtime) and system library, the hardware abstraction layer (HAL layer), and the kernel layer (Linux Kernel, kernel layer).

[0221] Among them, the APP layer can include a series of application packages. Fig.9A As shown, the application package may include camera, game, video, live broadcast and other applications, which are not listed here one by one and are not limited in this application.

[0222] The FWK layer provides an application programming interface (API) and a programming framework for the applications in the APP layer. In some implementations, these programming interfaces and programming frameworks can be described as functions. Fig.9A As shown, the FWK layer may include functions such as content provider, keyboard and mouse sharing service, collaborative service, data transmission module, etc., which are not listed one by one here and this application does not impose any restrictions on this.

[0223] The functions of the collaborative service and data transmission modules in Android devices are the same as those in Windows devices, and will not be repeated here.

[0224] The keyboard and mouse sharing service in Android devices is different from the keyboard and mouse sharing service in Windows devices. It mainly includes functional modules, such as Fig. 9BAs shown, there are window management module, window setting module, protocol management module and virtual data injection module, etc. Among them, the functions of the window management module, window setting module and protocol management module are the same as those of the window management module, window setting module and protocol management module in Windows devices, and they will not be repeated here.

[0225] Since the user did not release the left button of the mouse and then press it again when the mouse passed from the Windows device to the Android device, the cursor 10b (such as Figure 3B As shown in the figure, there is no corresponding down event on the tablet side. In this way, when the user releases the mouse and the PC sends a window release instruction to the tablet, the tablet cannot detect the up event, thereby releasing the floating window 40a. In order to enable the tablet to release the floating window 40a, in this embodiment, the keyboard and mouse sharing service in the tablet will actively register a down event through the virtual data injection module when the mouse crosses. In this way, when the user subsequently releases the left mouse button, the up event will be detected.

[0226] Continue to see Fig.9A For example, Android Runtime includes a core library and a virtual machine. Android Runtime is responsible for scheduling and management of the Android system.

[0227] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0228] The APP layer and FWK layer run in the virtual machine. The virtual machine executes the java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0229] Continue to see Fig.9A For example, the system library may include multiple functional modules, such as a surface manager, a media library, a three-dimensional (3D) graphics processing library (such as OpenGL ES), a two-dimensional (2D) graphics engine (such as SGL), etc.

[0230] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications. The media library supports multiple commonly used audio and video format playback and recording, as well as static image files. The media library can support multiple audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing.

[0231] It can be understood that the 2D graphics engine mentioned above is a drawing engine for 2D drawing.

[0232] Continue to see Fig.9A For example, the HAL layer is an interface layer between the operating system kernel (kernel layer) and the hardware circuit. Its purpose is to isolate the FWK layer from the kernel so that Android does not rely too much on the kernel, so that the development of FWK can be carried out without considering the driver.

[0233] Continue to see Fig.9A For example, the HAL layer may include various interfaces, such as audio and video interfaces, USB interfaces, Bluetooth interfaces, WiFi interfaces, etc., which are not listed here one by one and are not limited in this application.

[0234] Continue to see Fig.9A , exemplarily, the kernel layer in the Android system is a layer between hardware and software. The kernel layer may include various processes / threads, a power management module, and various drivers, such as a WiFi driver, a Bluetooth driver, and a kernel driver (Uinput driver) for simulating user input events.

[0235] This is the end of the introduction to the software architecture of Android devices. It is understandable that: Fig.9A and Fig. 9B The layers in the software Android device shown and the functional modules contained in each layer do not constitute a specific limitation on the Android device. In some possible implementations, the Android device may include more or fewer layers than shown, and each layer may include more or fewer components, which is not limited in this application.

[0236] In order to better understand the specific implementation of the cross-device dragging method of windows provided by the present application, the following, in combination with the accompanying drawings, describes in detail the interaction logic of the keyboard and mouse sharing service (including functional modules), collaborative services, and data transmission modules in Windows devices, and the keyboard and mouse sharing service (including functional modules), collaborative services, and data transmission modules in Android devices.

[0237] See also Fig.10 In order to implement the cross-device dragging method of the window provided in the application, the system message module, window management module, window setting module, protocol management module, collaborative service, and data transmission module in the Windows device need to cooperate with each other. Fig.10 The interactive processing is performed in the order of steps 101 to 139 shown. The data transmission module, protocol management module, window setting module, virtual data injection module, window setting module, and collaborative service in the Android device need to be performed in accordance with Fig.10 The interactive processing is performed in the order of steps 201 to 227 shown. For the sequential logic between the operations processed by the Windows device and the operations processed by the Android device, please refer to Fig.10 .

[0238] To facilitate the distinction between functional modules in Windows devices and Android devices, in the following embodiments, the system message module, window management module, window setting module, protocol management module, collaborative service, and data transmission module in Windows devices are represented as Windows system message module, Windows window management module, Windows window setting module, Windows protocol management module, Windows collaborative service, and Windows data transmission module, respectively; the window management module, window setting module, protocol management module, collaborative service, data transmission module, and virtual data injection module in Android devices are represented as Android window management module, Android window setting module, Android protocol management module, Android collaborative service, Android data transmission module, and Android virtual data injection module, respectively.

[0239] against Fig.10 The specific details of steps 101 to 139 implemented by the Windows device shown in FIG. Fig.11 It should be noted that Fig.11 Only part of the text is shown, and specific details can be found in the corresponding text part.

[0240] See also Fig.11 In order to implement the cross-device dragging method of the window provided in the application, the system message module, the window management module, the window setting module, the protocol management module, the collaborative service, and the data transmission module in the Windows device perform steps 101 to 139, specifically including:

[0241] 101. The Windows window management module sends a request for registering a window movement message fence to the Windows system message module in advance, so that the Windows message system can monitor the movement of the window displayed in the Windows device.

[0242] 102. The Windows window management module sends a request for registering a mouse message fence to the Windows system message module in advance, so that the Windows message system can monitor the mouse used in the keyboard and mouse sharing scenario.

[0243] Specifically, in this embodiment, the cross-device dragging of the window depends on the keyboard and mouse sharing function, that is, the Android device to which the window in the Windows device is dragged has established a keyboard and mouse sharing connection with the Windows device. Therefore, in a feasible implementation, the operations of step 101 and step 102 can be triggered and executed after the Windows device and the Android device have established a keyboard and mouse sharing connection.

[0244] Exemplarily, when a user uses a mouse in a keyboard and mouse sharing scenario to drag a window displayed in a Windows device (taking window 20a in the above embodiment as an example), the window move message fence and mouse message fence registered in steps 101 and 102 will be triggered. In this case, the Windows system message module will execute steps 103 and 104.

[0245] 103. The Windows system message module obtains a window move message corresponding to the window 20a, and sends the window move message to the Windows window management module.

[0246] 104. The Windows system message module obtains a message that the mouse used in the keyboard and mouse sharing scenario is pressed, determines that the left button of the mouse is currently pressed, and sends the left button of the mouse press message to the window management module of the Windows device.

[0247] 105. After receiving both the window move message sent by the Windows system message module and the mouse left button press message sent by the Windows system message module, the Windows window management module determines that the window 20a is currently being dragged and moved.

[0248] It should be noted that, in some possible implementations, the dragged window 20a may be displayed in the Windows device in a maximized mode, that is, the window 20a occupies the entire display area of ​​the Windows device. In this case, when it is confirmed that the window 20a is dragged and moved, the Windows device responds to the dragging operation and automatically restores the window 20a from the maximized mode to the small window mode, and then moves it as the mouse is dragged.

[0249] In addition, it should be noted that in some other possible implementations, the dragged window 20a may be currently displayed in a small window mode, such as Figure 2A In this case, when the confirmation window 20a is dragged and moved, the window 20a maintains the current style and moves with the mouse.

[0250] In addition, it should be noted that, since the current Windows device and the Android device are in a keyboard and mouse sharing scenario, when the user uses the mouse to drag the window 20a, it is possible that the window 20a displayed in the Windows device needs to be dragged to the Android device for display. Therefore, when the Windows window management module determines that the window 20a is currently being dragged and moved, it will send the window parameters of the window 20a to the Windows window setting module for subsequent processing, that is, executing step 106.

[0251] In addition, it should be noted that, in one possible case, the user may use the arrow keys and shortcut keys on the keyboard to control the movement of window 20a on the screen of the Windows device. In this case, since the user does not operate the mouse, only the window move message fence registered in step 101 is triggered. Therefore, the Windows system message module only executes step 103, that is, sending the window move message to the Windows window management module for processing. Since there is only a window move message and no left mouse button press message, the Windows window management module determines that window 20a is not currently being dragged and moved. In this case, the Windows window management module will not execute step 106.

[0252] In addition, in another possible case, the user may only press the left mouse button to click on a control in the window 20a, or click on other icons or documents on the desktop. In this case, since the window 20a has not moved, the Windows system message module only executes step 104, that is, sends the left mouse button pressing message to the Windows window management module for processing. Since there is only the left mouse button pressing message and no window moving message corresponding to the window 20a, the Windows window management module determines that the window 20a is not currently being dragged and moved. In this case, the Windows window management module will not execute step 106.

[0253] In addition, in another possible case, the user may press the right button of the mouse. In this case, since the window 20a does not move, the Windows system message module only executes step 104, that is, sends the right button of the mouse press message to the Windows window management module for processing. Since there is only the right button of the mouse press message, there is no left button of the mouse press message, and there is no window movement message corresponding to the window 20a, the Windows window management module determines that the window 20a is not currently being dragged and moved. In this case, the Windows window management module will not execute step 106.

[0254] It should be understood that the above description is merely an example listed for a better understanding of the technical solution of this embodiment, and is not intended to be the sole limitation to this embodiment.

[0255] 106. The Windows window management module sends the relevant window parameters of the window 20a to the Windows window setting module.

[0256] Among them, the relevant window parameters of window 20a are, for example, the handle of window 20a. It can be understood that the handle of the window includes the starting point (vertex) coordinates of the upper left corner of the window, as well as the width data and height data of the window. Therefore, by obtaining the handle of window 20a, the size information of window 20a (width data xWidth, height data yHeight) can be determined.

[0257] 107. The Windows window setting module calculates the window data (such as width data xWidth and height data yHeight) of the window 20a according to the window parameters of the window 20a sent by the Windows window management module.

[0258] It can be seen from the description of the above embodiment that in the cross-device dragging method of the window provided in the present application, the crossing of the window 20a occurs when the mouse crosses. Therefore, after the Windows system message module monitors the window movement message and the left mouse button pressing message of the window 20a, if the user continues to press the left mouse button to drag the window 20a, such as moving the window 20a to the side edge of the screen, such as the right edge in the above embodiment, as the cursor displayed on the Windows device, such as the cursor 10a in the above embodiment, moves, the Windows system message module will continue to obtain the mouse movement position information, and send the mouse movement position information to the Windows window management module, that is, execute step 108.

[0259] It should be noted that in this embodiment and the following embodiments, the mouse movement position information refers to the cursor corresponding to the mouse, such as the movement position information of the cursor 10a in the above embodiment.

[0260] 108. The Windows system message module sends the mouse movement position message to the Windows window management module.

[0261] 109. The Windows window management module performs edge detection according to the mouse movement position information to determine whether the mouse is dragged to the side edge of the Windows device screen (such as the right edge in the above embodiment).

[0262] Understandably, for Windows devices, the size of the screen is known. Therefore, when the size of the screen is known and the current mouse movement position information (coordinates) of the cursor 10a is also known, it is possible to determine whether the mouse is dragged to the side edge of the Windows device screen.

[0263] Specifically, in the keyboard and mouse sharing scenario, when the Windows window management module determines that the mouse is dragged to the edge of the screen (the left edge or the right edge), the keyboard and mouse crossing will be triggered. At the same time, in the cross-device dragging solution of the window provided by the present application, the Windows window management module will notify the Windows window setting module that the drag crossing has occurred, that is, execute step 110, so that the Windows window setting module can generate a window crossing instruction (command) according to the window data, that is, execute step 111.

[0264] 110. When the Windows window management module determines that the mouse has reached the edge of the screen according to the mouse movement position information, the Windows window setting module is notified that a drag crossing has occurred.

[0265] 111. The Windows window setting module generates a window crossing instruction corresponding to the window 20a according to the window data, and sends the window crossing instruction of the window 20a to the Windows protocol management module.

[0266] 112. The Windows protocol management module packages the data of the window crossing indication according to a predetermined protocol.

[0267] 113, the Windows protocol management module sends the obtained window crossing indication data packet to the Android data transmission module via the Windows data transmission module.

[0268] Understandably, whether it is the Windows protocol management module or the Android protocol management module, it is mainly responsible for the packaging and unpacking of the relevant instructions (commands), window parameters, confirmation information, etc. generated during the window dragging process. That is, the data that needs to be sent to the other end needs to be packaged by the protocol management module of the current device. Then it is sent to the data transmission module of the other end by the data transmission module of the current device.

[0269] Correspondingly, the data packets from the other end also need to be unpacked by the protocol management module of the current device before they can be sent to the corresponding functional module for subsequent processing.

[0270] The protocols followed by the Windows protocol management module for package assembly and unpacking, and the Android protocol management module for unpacking and package assembly are pre-agreed by the Android device and the Windows device. The specific agreement method and the predetermined protocol are not limited by this application.

[0271] See also Fig.11 It can be seen that the subsequent processing of dragging window 20a in the Windows device can only be continued after the Windows data transmission module receives the window crossing confirmation data packet fed back by the Android data transmission module. The Android data transmission module sends the window crossing confirmation data packet to the Windows data transmission module, which needs to go through Fig.10 The processing of steps 201 to 207 shown in FIG. For specific implementation details of steps 201 to 207, please refer to Fig.15 ,and Fig.15 The corresponding embodiment part will not be described in detail here.

[0272] 114. After receiving the window crossing confirmation data packet sent by the Android data transmission module, the Windows data transmission module sends the window crossing confirmation data packet to the Windows protocol management module for unpacking processing.

[0273] 115. The Windows protocol management module unpacks the window crossing confirmation data packet according to the agreed protocol, and then obtains a window crossing confirmation message.

[0274] 116. The Windows protocol management module sends the unpacked window crossing confirmation message to the Windows window setting.

[0275] 117. After receiving the window crossing confirmation message for window 20a fed back by the Android device, the Windows window setting module hides window 20a displayed on the screen of the Windows device.

[0276] Specifically, the window 20a is hidden at the specified coordinates, so that when the window 20a is subsequently dragged from the Android device back to the Windows device, the window 20a will appear from the hidden coordinates.

[0277] Alternatively, after the collaborative connection between the Windows device and the Android device is disconnected, the window 20a may automatically appear from the hidden coordinates. Alternatively, after the collaborative connection between the Windows device and the Android device is disconnected, when the user clicks the icon of the document 2 displayed on the Windows device, the window 20a appears from the hidden coordinates in response to the operation.

[0278] It should be understood that the above description is merely an example listed for a better understanding of the technical solution of this embodiment, and is not intended to be the sole limitation to this embodiment.

[0279] In addition, it should be noted that the display process of the window 20a in the PC can be divided into synthetic rendering, caching and display. Among them, the synthetic rendering of the picture corresponding to the window 20a (hereinafter referred to as the window 20a) is, for example, Fig. 8A After the synthesized and rendered window 20a is obtained, it is cached in the cache corresponding to the window 20a, and then taken out from the cache through the display driver for display.

[0280] Specifically, for composite drawing canvas, it is usually larger than the display area of ​​the Windows device screen. Fig.12 As shown, the composite drawn canvas can have a visible area (the display interface of the Windows device screen) and an invisible area.

[0281] In the scenario where the window 20a needs to be displayed on the Windows device screen, after the display driver takes the window 20a out of the cache, the window 20a is sent to the display Fig.12 The visible area of ​​the canvas, that is, the display interface of the Windows device screen, is displayed.

[0282] In the scenario where the window 20a does not need to be displayed on the Windows device screen, such as the scenario of hiding the window 20a in this embodiment, the window 20a is still being synthesized and drawn, but the display driver takes the window 20a from the cache and sends it to the display. Fig.12 The invisible area of ​​the canvas.

[0283] It should be understood that the above description is merely an example listed for a better understanding of the technical solution of this embodiment, and is not intended to be the sole limitation to this embodiment.

[0284] In addition, in some possible implementations, in order to provide users with a better user experience, by determining the hidden coordinates, and then determining the proportional relationship between the y-axis coordinates of the hidden coordinates and the height yHeight of the Windows device, the side edge of the Android device, such as Figure 3C The left edge shown in the figure passes through the floating window 40a at a corresponding proportion.

[0285] It should be understood that the above description is merely an example listed for a better understanding of the technical solution of this embodiment, and is not intended to be the sole limitation to this embodiment.

[0286] In order to better understand the determination of the hidden coordinates of the window 20a, the following is combined with Fig.13 , Fig.14A and Fig. 14B Provide explanation.

[0287] See also Fig.13 The determination of the hidden coordinates is mainly implemented by the Windows window setting module, and can be specifically implemented through steps S1 to S5.

[0288] S1, the Windows window setting module determines the width data xWidth and the height data yHeight of the window 20a according to the window parameters.

[0289] Specifically, the Windows window setting module determines the width and height of the window according to the current handle of the window 20a provided by the Windows window management module. For implementation details on determining the width and height of the window according to the handle of the window, please refer to existing documents and will not be described here.

[0290] S2, the Windows window setting module determines the current coordinates (x0, y0) of the cursor 10a according to the mouse movement position information.

[0291] S3, Windows window determines x0≤x_min / x0≥x_max.

[0292] Exemplarily, when the top left vertex of the Windows device screen is the origin coordinate (0, 0), x_min is 0. x_max is the coordinate corresponding to the X axis in the coordinates of the bottom right vertex of the screen. For example, for a 1920*1080 screen, x_max is 1920.

[0293] Based on this, if x0 ≤ x_min, it indicates that the mouse crosses from the left edge of the Windows device screen; if x0 ≥ x_max, it indicates that the mouse crosses from the right edge of the Windows device screen; if x_min < x0 < x_max, it indicates that the mouse is still within the Windows device screen, has not reached the left edge, and has not reached the right edge, that is, no crossing will occur.

[0294] When it is determined that no crossing occurs, the Windows window setting module can wait for the window parameters and mouse movement position information obtained at the next moment, and then re-execute step S3 according to the newly obtained information.

[0295] S4, the Windows window determines that the dragged window 20a crosses from the left edge / right edge of the screen.

[0296] S5, the Windows window determines the hidden coordinates (x, y) of the window 20a in the Windows device according to the crossing position and the current coordinates of the cursor 10a.

[0297] It can be understood that in this embodiment, the crossing of the keyboard and mouse and the window are all from the left edge or the right edge of the Windows device screen. Therefore, the Y-axis coordinate of the hidden coordinate can directly be the current Y-axis coordinate of the cursor 10a, such as y0 determined in step S2. The X-axis coordinate needs to be determined according to the crossing position combined with the width data xWidth.

[0298] See Fig.14A , exemplarily, for a scenario where the screen width and height of the Windows device are 1920*1080 and the width and height of the window 20a are 200*100, during the process of the user dragging the window 20a to the right edge of the Windows device screen, when the Windows window setting module determines that x0 ≥ 1920 according to the Fig.13 processing flow, it can be determined that the window 20a and the mouse will cross from the right edge of the Windows device screen. For the Fig.14A shown scenario, the coordinate in the X-axis direction of the hidden coordinate is x = Xright = x_max = 1920, and the coordinate in the Y-axis direction is y = Yright = y0. That is, the hidden coordinate (x, y) is (1920, y0).

[0299] See Fig. 14BFor example, for a scenario where the screen width and height of a Windows device is 1920*1080 and the width and height of window 20a is 200*100, when the user drags window 20a to the left edge of the Windows device screen with a mouse, when the Windows window setting module follows Fig.13 When x0≤0 is determined in the processing flow, it can be determined that the window 20a and the mouse will cross the left edge of the Windows device screen. Fig. 14B In the scenario shown, this embodiment is set to hide the window 20a only when it is completely moved out of the Windows device screen. Therefore, the coordinates of the hidden coordinates in the X-axis direction are x=Xleft=-xWidth, and the coordinates in the Y-axis direction are y=Yleft=y0. That is, the hidden coordinates (x, y) are (-xWidth, y0).

[0300] It should be understood that the above description is merely an example listed for a better understanding of the technical solution of this embodiment, and is not intended to be the sole limitation to this embodiment.

[0301] It can be seen from the description of the above embodiment that when the window 20a crosses, the first activated floating window 40a is not directly displayed in the middle of the Android device screen, but is displayed on the edge of the screen close to the Windows device side, such as the floating window 40a of one third of the size mentioned above (such as Figure 3C Therefore, even if crossing occurs, before the user releases the mouse, that is, releases the pressing of the left mouse button, the user can still press the left mouse button to continue moving.

[0302] Exemplary, in some possible situations, the user may drag the window 20a back to the Windows device before releasing the mouse. In other possible situations, for the scene crossing from the right edge, it may continue to move to the right so that the floating window 40a can be fully displayed on the Android device screen.

[0303] Therefore, when the left mouse button continues to be pressed and moved, the Windows system message module will continue to obtain the mouse movement position information and send the mouse movement position information to the Windows window management module for processing, that is, executing step 118.

[0304] 118. The Windows system message module obtains the mouse movement position information and sends the mouse movement position information to the Windows window management module.

[0305] 119. The Windows window management module determines the window movement position information according to the mouse movement position information, and sends the window movement position information to the Windows window setting module.

[0306] Understandably, since the mouse cursor is on the window during the window dragging process, the window is moved. Therefore, when the mouse movement position information is known, the Windows window management module can directly determine the window movement position information corresponding to the window 20a.

[0307] 120. The Windows window setting module generates a window movement instruction according to the window movement position information sent by the Windows window management module, and sends the window movement instruction to the Windows protocol management module.

[0308] 121. The Windows protocol management module packages the window movement indication data according to the agreed protocol, and then obtains the window movement indication data packet.

[0309] 122, the Windows protocol management module sends the obtained window movement indication data packet to the Android data transmission module via the Windows data transmission module.

[0310] In this way, the Android window device module can change the position of the floating window 40a according to the window movement position information.

[0311] Since the user can release the mouse at any time after the floating window 40a appears on the Android device screen, during the execution of steps 118 to 122, the operations of steps 123 to 126 may also occur.

[0312] Continue to see Fig.11 For example, when the user releases the mouse, the Windows system message module will monitor the left mouse button lifted message and the window stopped moving message, and send the left mouse button lifted message and the window stopped moving message to the Windows window management module for subsequent processing, that is, executing steps 123 and 124.

[0313] 123. The Windows system message module sends a message indicating that the left button of the mouse has been lifted to the Windows window management module.

[0314] 124. The Windows system message module sends a window stop moving message to the Windows window management module.

[0315] 125. After receiving the left mouse button lifted message and the window movement stopped message sent by the Windows system message module, the Windows window management module generates a window release instruction and sends the window release instruction to the Windows window device setting module.

[0316] 126. After receiving the window release instruction sent by the Windows window management module, the Windows window device setting module sends the window release instruction to the Windows protocol management module for packet processing.

[0317] 127, the Windows protocol management module packages the window release indication data according to the agreed protocol, and then obtains the window release indication data packet.

[0318] 128. The Windows protocol management module sends the obtained window release indication data packet to the Windows data transmission module, which then sends it outward.

[0319] 129, the Windows data transmission module sends the window release indication data packet provided by the Windows protocol management module to the Android data transmission module, so that other functional modules in the Android device perform subsequent processing according to the window release indication data packet.

[0320] It can be seen from the description of the above embodiments that the cross-device dragging of windows provided in this application also requires the use of the collaborative function to realize the dragging of the window 20a displayed on the Windows device screen to the Android device screen for display. In addition, it can be seen from the description of the above embodiments that when the user releases the mouse, that is, when the loading icon 40a-5 appears on the floating window 40a, the Windows device and the Android device will automatically turn on the collaborative function without the user's perception and without the need for the user to turn on the collaborative function, and establish a collaborative connection between the two in an extended mode supported by the collaborative function (hereinafter referred to as an extended connection). Therefore, while generating a release window indication, the Windows window management module will execute the operation of turning on the collaborative function, that is, execute step 130.

[0321] 130. When generating a window release instruction, the Windows window management module notifies the Windows collaborative service to start a collaborative function.

[0322] 131. The Windows collaborative service sends an extended mode indication to the Windows protocol management module so that the Windows collaborative management module packages the extended mode indication.

[0323] 132. The Windows protocol management module packages the data indicated by the extended mode according to the agreed protocol, thereby obtaining an extended mode indication data packet.

[0324] 133, the Windows protocol management module sends the obtained extended mode indication data packet to the Windows data transmission module, which then sends it outward.

[0325] 134, the Windows data transmission module sends the extended mode indication data packet provided by the Windows protocol management module to the Android data transmission module, so that other functional modules in the Android device perform subsequent processing according to the extended mode indication data packet, thereby completing the extended connection between the two.

[0326] In some embodiments, Fig.10 As shown in the figure, the Android device needs to go through at least 10 steps from receiving the extended mode indication packet to sending the extended connection success packet to the PC. Fig.10 Only by performing the processing from step 222 to step 227 shown in the figure can the establishment of the extended connection between the Windows device and the Android device be realized. For the specific implementation details of step 222 to step 227, please refer to Fig.15 ,and Fig.15 The corresponding embodiment part will not be described in detail here.

[0327] 135. After receiving the extended connection success data packet sent by the Android data transmission module, the Windows data transmission module sends the extended connection success data packet to the Windows protocol management module for unpacking processing.

[0328] 136. The Windows protocol management module unpacks the extended connection success data packet according to the agreed protocol, and obtains an extended connection success message fed back by the Android device.

[0329] 137. The Windows protocol management module sends an extension connection success message to the Windows window setting module.

[0330] 138. After confirming that the extended connection between the Windows device and the Android device is successful, the Windows window setting module will generate a collaborative screen that needs to be displayed in the display area of ​​the floating window 50a and send it to the Android device for display.

[0331] The generation of the collaborative screen is similar to that of the window 20a, except that when generating the collaborative screen, the synthetic rendering process is performed based on the maximized window 20a and the taskbar 30a, so the generated collaborative screen ultimately includes the above-mentioned window 20b and taskbar 30b.

[0332] For example, the synthetic rendering of the collaborative screen is still in Fig. 8AAfter obtaining the synthesized and rendered collaborative screen, it is cached in the cache corresponding to the collaborative screen, and then packaged according to the agreed protocol through the Windows protocol management module, and the obtained collaborative screen package is sent to the Android data transmission module through the Windows data transmission module.

[0333] Correspondingly, after receiving the package of the collaborative screen, the Android data transmission module will forward it to the Android protocol management module for unpacking, and then obtain the collaborative screen, and finally display the obtained collaborative screen in the display area of ​​the floating window 50a.

[0334] For details about the specific contents of the collaborative screen, see the above Figure 3E The description of the corresponding embodiments will not be repeated here.

[0335] After completing the operation of step 138 , the Windows setting module will notify the Windows window management module that the dragging of window 20 a is completed, and then execute step 139 .

[0336] At this point, the window 20a displayed on the Windows device screen is dragged to the Android device screen, and the corresponding screen changes are shown in the figure below. Figure 2A , Figure 2B , Figure 3A to Figure 3D .

[0337] This is the end of the introduction to the operations performed by the functional modules in the Windows device during the cross-device dragging of the window. Parts not described in this embodiment can be understood in conjunction with the operations performed by the functional modules in the Android device, and will not be repeated here.

[0338] against Fig.10 The specific details of steps 201 to 227 implemented by the Android device shown in Fig.15 It should be noted that Fig.15 Only part of the text is shown, and specific details can be found in the corresponding text part.

[0339] See also Fig.15 In order to implement the cross-device dragging method of the window provided in the application, the data transmission module, the protocol management module, the window setting module, the virtual data injection module, the window setting module, and the collaborative service in the Android device perform steps 201 to 227, specifically including:

[0340] 201. After receiving the window crossing indication data packet sent by the Windows data transmission module, the Android data transmission module sends the window crossing indication data packet to the Android protocol management module for unpacking.

[0341] 202, the Android protocol management module unpacks the window crossing indication data packet according to the agreement, and then obtains the window crossing indication.

[0342] 203, the Android protocol management module sends the obtained window crossing indication to the Android window management module for processing.

[0343] 204. The Android window management module starts the floating window 40a according to the window crossing instruction.

[0344] In this way, before the extended connection between the Windows device and the Android device is successfully established, the floating window 40a can be used to replace the floating window 50a, so that the user visually feels that the window 20a on the Windows device has indeed been dragged to the Android device and is just waiting for the content of the window 20a to be loaded.

[0345] 205, the Android window management module generates a window crossing confirmation message, and sends the window crossing confirmation message to the Android protocol management module for packaging.

[0346] 206 , the Android protocol management module packages the window crossing confirmation data according to the agreed protocol, and then obtains a window crossing data confirmation data packet.

[0347] 207, the Android protocol management module sends the obtained window crossing data confirmation data packet to the Windows data transmission module via the Android data transmission module.

[0348] In this way, after receiving the window traversal data confirmation packet sent by the Android device, the Windows device can execute Fig.11 Step 114 and subsequent steps shown in FIG.

[0349] Continue to see Fig.15 , exemplarily, after starting the floating window 40a, the Android window management module will continue to execute steps 208 to 211. That is, steps 208 to 211 can be performed simultaneously with steps 205 to 208, regardless of order.

[0350] 208. After starting the floating window 40a, the Android window management module determines the display position of the floating window 40a on the Android device screen, and sends the determined display position information to the Android window setting module to instruct the Android window setting module to set the display position of the floating window 40a.

[0351] 209. The Android window setting module sets the display position of the floating window 40a according to the display position information provided by the Android window setting module, so that the floating window 40a can be displayed at the specified position of the Android screen.

[0352] It should be understood that if the mouse does not have a corresponding down event, when the mouse crosses to the Android device and the user releases the mouse, an up event of the mouse will occur, so that the Windows device cannot perceive it. Before the mouse crosses, the user's pressing operation on the left mouse button (down event) is registered on the Windows device. When the mouse crosses from the Windows device to the Android device, there is no corresponding down event in the Android device. Therefore, in order to ensure that the Windows device can be informed when the mouse is released after the mouse crosses to the Android device, the Android window setting module will send the position information of the cursor 10b currently corresponding to the mouse on the Android device to the Android virtual data injection module after setting the display position of the floating window 40a, so that the Android virtual data injection module can inject a down event for the mouse on the Android device. That is, execute steps 210 and 211, so that the Windows device will perceive that the mouse is released.

[0353] 210, the Android window setting module informs the Android virtual data injection module of the mouse position of the mouse on the Android device, that is, the position information of the cursor 10b corresponding to the mouse on the Android device.

[0354] 211, the Android virtual data injection module injects a down event at the mouse position.

[0355] pass FIG. 3A to FIG. 3EFrom the description of the illustrated embodiment, it can be seen that when the mouse and window 20a cross, and the floating window 40a is displayed on the Android device screen for the first time, the floating window 40a is not displayed completely on the Android device screen. Instead, the floating window 40a is displayed at the side edge of the screen close to the Windows device, such as the left edge in the above embodiment, which is about one-third the size. Therefore, in order to still be able to drag the floating window 40a to move after the mouse crosses to the Android device, it is necessary to inject the corresponding down event of the mouse at the position where the mouse cursor 10b is displayed on the status bar 1 of the floating window 40a, and to monitor the dragging of the status bar 1. For the implementation of this operation, please refer to Fig.16A The operations performed by the window setting module and HwFreeFormCaptionView in the illustrated embodiment are not described in detail here.

[0356] In addition, through the above description of the operation of the functional module in the Windows device, it can be known that after the Windows device receives the window crossing confirmation data packet fed back by the Android device, as the user continues to press the left mouse button and move the mouse, a window movement indication data packet will be generated and sent to the Android device. After receiving the window movement indication data packet, the Android device will perform the operations of step 212 to step 216.

[0357] 212. After receiving the window movement instruction data packet sent by the Windows data transmission module, the Android data transmission module sends the window movement instruction data packet to the Android protocol management module for unpacking.

[0358] 213, the Android protocol management module unpacks the window movement instruction data packet according to the agreement, and then obtains the window movement instruction.

[0359] 214. The Android protocol management module sends the obtained window movement instruction to the Android window management module for processing.

[0360] 215. The Android window management module extracts the window movement position information from the window movement indication, and sends the extracted window movement position information to the Android window setting module.

[0361] 216. The Android window setting module moves the position of the floating window 40a displayed on the Android screen according to the window movement position information sent by the Android window management module.

[0362] For example, according to the window moving position information, the floating window 40a can be moved from Figure 3C Move to the position shown in Figure 3D The location is shown.

[0363] Understandably, before the user releases the mouse, as the mouse is pressed and moved, the Windows device will continue to send window movement indication data packets corresponding to each movement to the Android device, and the Android device will continuously move the display position of the floating window 40a according to the window movement position information obtained by unpacking.

[0364] From the above description, it can be seen that after the Android virtual data injection module injects a down event into the mouse, when the user releases the mouse, the Windows device can sense it, and then the above functional modules inside the Windows device are processed, and finally the Windows data transmission module sends the release window indication data packet and the extended mode indication data packet to the Android data transmission module. Fig.10 and Fig.15 It can be seen that the Android device will execute steps 217 to 221 after receiving the release window indication data packet, and will execute steps 222 to 227 after receiving the extended mode indication data packet.

[0365] 217, after receiving the release window indication data packet sent by the Windows data transmission module, the Android data transmission module sends the release window indication data packet to the Android protocol management module for unpacking.

[0366] 218, the Android protocol management module unpacks the release window indication data packet according to the agreement, and then obtains the release window indication.

[0367] 219, the Android protocol management module sends the obtained release window instruction to the Android window management module for processing.

[0368] 220. The Android window management module sends a window release instruction to the Android window setting module, so that the Android window setting module can release the floating window 40a.

[0369] 221. The Android window setting module releases the floating window 40a.

[0370] At this time, the control 40a-5 can be displayed in the floating window 40a.

[0371] It should be noted that, since the mouse down event is injected in step 211, when the mouse is released and a window release instruction is received from the Windows device, the Android device can inject the mouse up event to release the floating window 40a.

[0372] 222, after receiving the extended mode indication data packet sent by the Windows data transmission module, the Android data transmission module sends the extended mode indication data packet to the Android protocol management module for unpacking.

[0373] 223, the Android protocol management module unpacks the extended mode indication data packet according to the agreement, and then obtains the extended mode indication.

[0374] 224, the Android protocol management module sends the obtained extended mode indication to the Android collaborative service for processing.

[0375] Since the time interval between the release window indication and the extended mode indication reaching the Android device is usually short, the Android device is unpacking and transmitting the extended mode indication data packet during the process of releasing the floating window 40a. Therefore, after the Android window setting module releases the floating window 40a, the Android collaborative service will quickly perform an extended connection with the Windows device and generate an extended connection success message.

[0376] 225, the Android collaborative service performs an extended connection with the Windows device, and after completing the extended connection, generates an extended connection success message, and sends the extended connection success message to the Android protocol management module for packaging.

[0377] 226, the Android protocol management module packages the data of the successful extension connection according to the agreed protocol, and then obtains a successful extension connection data packet.

[0378] 227, the Android protocol management module sends the obtained extended connection success data packet to the Android data transmission module, and then the Android data transmission module sends the extended connection success data packet to the Windows data transmission module.

[0379] At this point, the window 20a displayed on the Windows device screen is dragged to the Android device screen, and the corresponding screen changes are shown in the figure below. Figure 2A , Figure 2B , Figure 3A to Figure 3D About Figure 3D The floating window 40a shown is replaced by Figure 3E The details of the floating window 50a are shown below. Fig.16A Provide detailed explanation.

[0380] From the description of the above embodiment, it can be known that the floating window 50a displayed on the screen of the Android device can be enlarged or reduced by dragging, and the size will also change after the screen is rotated. The realization of these functions requires relying on some functional modules (methods / functions) of the FWK layer in the Android system, such as ActivityStarter, DisplayContent, MultiWindowsManager, FreeFormCaptionView, MouseManager, etc.

[0381] Among them, ActivityStarter is a functional module for starting Activity components, such as floating window 40a and floating window 50a. Different from Android's native ActivityStarter, the ActivityStarter in this application can also set the collaborative mode corresponding to the floating window 50a. For the sake of distinction, the ActivityStarter implemented in this application will be represented by HwActivityStarterEx. It should be understood that this is only for the convenience of explanation, and the name of the functional module that implements this function is not limited.

[0382] Among them, DisplayContent, i.e., the display content manager, is responsible for the process task of prohibiting the floating window 40a from being scaled in this application. In this way, it can be ensured that after the floating window 50a is started, it can completely cover the floating window 40a without the user notifying it, and the floating window 50a will not be unable to be completely covered due to the enlargement of the floating window 40a.

[0383] Among them, MultiWindowsManager is a multi-window manager. Different from the native MultiWindowsManager of Android, the MultiWindowsManager in this application can be responsible for ensuring the effect of the floating window 50a in the horizontal screen state when the screen of the Android device is rotated in the vertical screen state. In order to distinguish, the MultiWindowsManager implemented in this application will be represented as HwMultiWindowsManager in the following. It should be understood that this is only for the convenience of explanation, and does not limit the name of the functional module that implements this function.

[0384] Among them, FreeFormCaptionView can be understood as the taskbar in free window mode, that is, the functional module responsible for managing the status bar 1 and status bar 2 in the floating window 40a and the floating window 50a. Because in the cross-device drag scheme of the window provided in this application, when the cursor 10b is placed in the status bar 1 area, the user can drag the floating window 40a and the floating window 50a by manipulating the mouse. Therefore, unlike the native FreeFormCaptionView of Android, the FreeFormCaptionView in this application can be used to set up drag monitoring for the taskbar. For the sake of distinction, the FreeFormCaptionView implemented in this application will be subsequently represented as HwFreeFormCaptionView. It should be understood that this is only for ease of explanation, and does not limit the name of the functional module that implements this function.

[0385] Among them, MouseManager, i.e. mouse management, is used to manage mouse events, such as down events, up events, drag events, etc. In this application, it is mainly used to manage the mouse drag monitoring of the status bar 1, and to release the mouse, and cancel the drag monitoring after the up event occurs. Since the Android device uses the keyboard and mouse of the Windows device in the keyboard and mouse sharing mode, the keyboard and mouse of the Windows device are also managed. For the sake of distinction, the MouseManager implemented in this application will be represented as HnPcMouseManager in the future. It should be understood that this is only for the convenience of explanation, and does not limit the name of the functional module that implements this function.

[0386] For ease of description, Fig.16A For the Android device side, only the collaborative service, the Android window management module and the Android window setting module in the keyboard and mouse service, as well as the HwActivityStarterEx, DisplayContent, HwMultiWindowsManager, HwFreeFormCaptionView and HnPcMouseManager listed above in the FWK layer are shown. For Windows devices, no specific functional modules are shown.

[0387] The window management module in the Android device can be represented as DragWindowManager, and the window setting module can be represented as PcScreenCoverActivity. It should be understood that this is only for the convenience of explanation, and the name of the functional module that implements the function is not limited.

[0388] See also Fig.16A, exemplarily, in this embodiment, the extended connection between the Windows device and the Android device is realized by triggering the collaborative function based on the keyboard and mouse sharing connection between the Windows device and the Android device. In order to ensure that after the extended connection between the Windows device and the Android device is established, the display area of ​​the floating window 50a can display the screen content of the window 20a recorded and delivered by the Windows device without black borders, when the Windows device and the Android device establish a keyboard and mouse sharing connection, the two agree on the aspect ratio of the area displaying the window 20a in the floating window 50a. In this way, the subsequent Windows device can record and deliver the window 20a according to the agreed aspect ratio, and the Android device can also scale the display in proportion according to the same aspect ratio, so that the display area of ​​the floating window 50a can display the window 20a without black borders, and the interface effect is more like directly dragging the window 20a to the Android device display.

[0389] Regarding the agreement on the aspect ratio, for example, when a Windows device and an Android device establish a keyboard and mouse sharing connection based on the keyboard and mouse sharing service, the Windows device determines the width and height of its current screen and sends it to the Android device. In this way, the Android device can know the width and height of the window 20a that needs to be maximized in the floating window if window crossing occurs later.

[0390] See also Fig. 16B For example, in a scenario where the aspect ratio of a Windows device is 16:9, the aspect ratio agreed upon with an Android device is also 16:9. However, for different forms of extended screen styles in an Android device, the aspect ratio is the ratio of different areas. For example, for a floating window 50a, an aspect ratio of 16:9 indicates the aspect ratio of the remaining area after removing status bar 1 and status bar 2 from the floating window 50a, that is, the ratio of the width of the floating window 50a: (height of the floating window 50a - height of status bar 1 - height of status bar 2) is 16:9.

[0391] Continue to see Fig. 16B , exemplarily, for a scenario where the extended screen is the entire screen of the Android device, the aspect ratio indication of 16:9 is the ratio of the area after removing the sidebar 50a-7 and the taskbar 30b.

[0392] It should be understood that the above description is merely an example listed for a better understanding of the technical solution of this embodiment, and is not intended to be the sole limitation to this embodiment.

[0393] Continue to see Fig.16AFor example, after a keyboard and mouse sharing connection is established between a Windows device and an Android device, and the aspect ratio is agreed upon, if the user presses the left mouse button and the mouse crosses when dragging the window, according to the above Figure 10 to Figure 13 , Fig.14A , Fig. 14B and Fig.15 The processing logic of the embodiment shown can determine the hidden coordinates of the window being dragged across. In order to ensure that the window that appears first on the Android device, such as the floating window 40a in the above embodiment, can reflect the effect of parallel crossing with the window 20a in the Windows device, the Y-axis height of the upper edge of the floating window 40a should be the same as the Y-axis coordinate of the hidden coordinate. Taking the Y-axis coordinate of the hidden coordinate as y0 as an example, Fig.14A and Fig. 14B shown.

[0394] Continue to see Fig.16A For example, after y0b is obtained by the Android window management module, the Android window management module will set the startup bounds of the floating window 40a according to y0 and the agreed aspect ratio, and set the window mode of the floating window 40a to the floating window mode.

[0395] It should be understood that floating window 40a is a virtual window that replaces floating window 50a on the Android device screen before floating window 50a starts to display. From the above description, it can be seen that it is not an extended screen of the full screen with floating window 50a, but a floating window mode, so the window mode of floating window 40 that replaces floating window 50a also needs to be set to floating window mode.

[0396] In addition, since the floating window 40a and the floating window 50a are generally the same in size, the area of ​​the floating window 40a that meets the agreed aspect ratio is the same as that of the floating window 50a. Fig. 16B The area shown in 1 is indicated.

[0397] Since the heights of the status bar 1 and the status bar 2 are usually fixed, and the widths are the same as the width of the area 1, therefore, when setting the bounds of the floating window 40a, the bounds of the area 1 may be determined first.

[0398] In addition, it should be noted that, in some possible cases, the height of status bar 1 and the height of status bar 2 may be the same.

[0399] In addition, since the floating window 40a is a virtual window that replaces the floating window 50a, in this embodiment, the floating window 40a and the floating window 50a are the same size as an example. In this case, whether the status bar 1 and the status bar 2 in the floating window 40a shown in the drawings, or the status bar 1 and the status bar 2 shown in the floating window 50a, their heights are the same.

[0400] Exemplary, in some possible implementations, such as when it is agreed that area 1 occupies 60% (or other percentage) of the Android device screen, you can first obtain the width / height of the Android device screen, and then divide the width / height of the Android device screen by 60% to get the width / height corresponding to area 1, and then according to the agreed aspect ratio and the determined width / height of area 1, you can get the height / width corresponding to area 1.

[0401] For example, after determining the width of region 1, the X coordinate of the left edge of region 1 (left) can be used as the X coordinate of the upper left corner vertex of floating window 40a, and the X coordinate of the right edge of region 1 (right) can be used as the X coordinate of the lower right corner vertex of floating window 40a.

[0402] In addition, it can be known from the above description that y0 is the Y-axis coordinate of the top edge (top) of the floating window 40a, so the Y-axis coordinate of the top left corner of the floating window 40a can be determined to be y0.

[0403] In addition, when the heights of status bar 1 and status bar 2 are fixed and the height of area 1 is determined, and top=y0 is known, the bottom edge (bottom) of floating window 40a is y0+height of status bar 1+height of area 1+height of status bar 2.

[0404] In this way, the setting of the bounds (left, top, right, bottom) of the floating window 40a can be completed.

[0405] For example, Fig.16A In the above method, the Android window management module sets the bounds and window mode of the floating window 40a, which can be used as Fig.15 A specific implementation form of step 208 in the illustrated embodiment.

[0406] Continue to see Fig.16A For example, after completing the above two steps (setting the bounds of the floating window 40a and setting the window mode to the floating window mode), the Android window management module will notify the Android window setting module to pull up the Activity of the floating window 40a, that is, create the Activity of the floating window 40a.

[0407] Understandably, when the Android window management module notifies the Android window setting module to pull up the Activity of the floating window 40a, it will inform the Android window setting module of the parameters set in the above two steps. In this way, when the Android window setting module calls the StartActivity method and notifies HwActivityStarterEx to draw the Activity of the floating window 40a, it will inform HwActivityStarterEx of the bounds and floating window mode of the floating window 40a as input parameters.

[0408] Continue to see Fig.16A For example, HwActivityStarterEx will draw the Activity of the floating window 40a according to the parameters passed by the Android window setting module, and notify HwFreeFormCaptionView to start the Caption in the floating window 40a, specifically to set a drag monitoring for the status bar 1 mentioned above, so that when the cursor 10b falls on the status bar 1, the user can control the mouse to drag the floating window 40a.

[0409] Continue to see Fig.16A For example, after HwFreeFormCaptionView sets the drag monitoring for the status bar 1, it will inform HnPcMouseManager. In this way, HnPcMouseManager can manage the drag monitoring of the status bar 1, such as monitoring whether there is an event of dragging the status bar 1.

[0410] In addition, it should be noted that HnPcMouseManager will cancel the drag monitoring of status bar 1 after the mouse is released and an up event occurs.

[0411] Continue to see Fig.16A For example, when HwActivityStarterEx draws the Activity of the floating window 40a, that is, when the Android device screen displays the floating window 40a, DisplayContent will also set the floating window 40a, specifically, to prohibit the floating window 40a from being scaled.

[0412] Continue to see Fig.16A , exemplarily, the Android window setting module will also avoid the hot zone of the status bar 1 of the floating window 40a when determining the floating window 40a, such as Figure 3D The area where the controls 40a-1 to 40a-4 are located is shown to set the initial position of the mouse, that is, the position of the cursor 10b after the mouse passes through.

[0413] Understandably, since the mouse has just passed through, the floating window 40a has just been displayed, that is, Figure 3C In the illustrated style, if the user does not release the mouse, the floating window 40a may continue to be dragged to the right, for example, to Figure 3D Therefore, the initial position set by the Android window management module for the cursor 10b specifically needs to fall within the area of ​​the status bar 1 and avoid various functional controls displayed on the status bar 1.

[0414] In addition, it should be noted that in some possible implementation scenarios, the area on the side edge of the Android device screen is usually set as a hot zone, that is, when the mouse is operated on the side edge, such as sliding, other windows or taskbars set will be pulled out. Therefore, the initial position set by the Android window management module for the cursor 10b also needs to avoid the hot zone on the side edge of the Android device screen. Since this case is to cross the left edge of the Android device screen to display the floating window 40a. Therefore, the initial position set by the Android window management module for the cursor 10b needs to avoid the hot zone on the left edge of the Android device screen.

[0415] Exemplarily, when the top of the floating window 40a is known and the height of the status bar 1 is known, in a possible implementation method, the X-axis coordinate of the initial position of the mouse can be between the left and right determined above, and avoid the coordinates of controls 40a-1 to 40a-4, represented by x1, and the corresponding y1 can be (top + height of status bar 1 / 2).

[0416] It should be understood that the above description is merely an example listed for a better understanding of the technical solution of this embodiment, and is not intended to be the sole limitation to this embodiment.

[0417] For example, Fig.16A The setting of the initial position of the mouse by the Android window device module (i.e., displaying the cursor 10b in the status bar 1) and the corresponding down event injected into the mouse, as well as the setting and cancellation of the drag monitoring of the status bar 1 by HwFreeFormCaptionView and HnPcMouseManager in the above manner, can be used as Fig.15 A specific implementation form of step 211 in the illustrated embodiment.

[0418] Continue to see Fig.16A For example, after setting the initial position of the mouse, the Android window setting module will inject a down event into the mouse. For details, see Fig.15 Steps 209 to 211 in the above are not described in detail here.

[0419] Continue to see Fig.16A For example, after completing the drawing and setting of the floating window 40a, the Android window setting module will call onResume() to inform the Android window management module that the floating window 40a has been started normally and is available.

[0420] Continue to see Fig.16A For example, if after completing the above operations, the Android window management module receives the mouse movement information sent by the Windows device, it can cooperate with other functional modules according to Fig.15 The operations from step 212 to step 216 shown in the figure are used to move the floating window 40a, and the specific implementation details are not repeated here.

[0421] Continue to see Fig.16A For example, if the user releases the mouse, that is, the Android device monitors the mouse up event, HnPcMouseManager will cancel the drag monitoring of the status bar 1. At the same time, based on Fig.10 and Fig.11 As shown, after the user releases the mouse, the Windows device will send a window release instruction to the Android device, and will also execute the operation of starting the extended mode of the collaborative function and establishing an extended connection with the Android device.

[0422] Continue to see Fig.16A For example, when the Android device establishes an extended connection with the Windows device, HwActivityStarterEx will first set the bounds of the floating window 50a when drawing the Activity of the floating window 50a.

[0423] From the above description, it can be known that floating window 50a needs to cover floating window 40a. Therefore, when setting the bounds of floating window 50a, directly find the bounds of the currently displayed floating window 40a, and then set the bounds of floating window 50a according to the bounds of floating window 40a.

[0424] Exemplarily, in a possible case, the bounds of the floating window 50a may be the same as the bounds of the floating window 40a. In this way, the floating window 40a can be seamlessly covered, and the user will not feel that the floating window 40a is replaced.

[0425] In addition, when drawing the Activity of the floating window 50 a , HwActivityStarterEx also needs to set a window module for the floating window 50 a .

[0426] Specifically, if there is a floating window 40a in the current interface of the Android device, the window mode of the floating window 50as is also set to the floating window mode. If there is no floating window 40a, the collaborative mode needs to be changed to the mirror mode, that is, the window mode of the floating window 50a is set to the full screen mode.

[0427] Continue to see Fig.16A For example, after completing the above settings and drawing the Activity of the floating window 50a according to the above requirements, that is, after starting the floating window 50a, HwActivityStarterEx can call finish() to close the floating window 40a.

[0428] For the process of releasing the mouse and establishing an extended connection, see Fig.11 Steps 125 to 137 are shown, and Fig.15 The specific implementation details of steps 217 to 227 are not repeated here.

[0429] Continue to see Fig.16A For example, after the floating window 50a is displayed, if the Android device screen is rotated, HwMultiWindowsManager will work to ensure that the display effect of the floating window 50a in the horizontal state can be guaranteed regardless of whether the Android device screen is in the horizontal state or the vertical state. For the changes in the position and size of the floating window 50a when the Android device screen is rotated, please refer to Figure 7 shown.

[0430] Understandably, if the Android device screen rotates during the process of displaying the floating window 40a, HwMultiWindowsManager can also ensure that the display effect of the floating window 40a in the horizontal state can be guaranteed regardless of whether the Android device screen is in the horizontal state or the vertical state.

[0431] This is the end of the introduction to the operations performed by the functional modules in the Android device during the cross-device dragging process of the window. Parts not described in this embodiment can be understood in conjunction with the operations performed by the functional modules in the Windows device, and will not be repeated here.

[0432] Therefore, when a keyboard and mouse sharing connection is established between a Windows device and an Android device, that is, the device is in a keyboard and mouse sharing scenario but the collaborative function is not started, when a user uses a Windows device, such as the PC mentioned in the above embodiment, to open multiple applications and needs to use these applications at the same time, such as viewing the contents in the windows corresponding to these applications, the cross-device dragging method of windows provided in this application can be used to monitor the dragging of the window corresponding to the application / document on the PC, and cooperate with the mouse crossing control in the keyboard and mouse sharing. When the mouse crosses, the position of the window moving out of the PC screen is calculated, and information related to the window is transmitted to the Android device, such as the tablet mentioned in the above embodiment. When the window is released on the tablet (the user releases the left mouse button), the PC and the tablet are triggered to connect collaboratively, and the tablet is set as an extended screen of the PC, and then the hidden window on the PC, such as the window of document 2 mentioned in the above embodiment, is moved to the extended screen, i.e., the tablet for display, thereby realizing a dual-screen office experience.

[0433] Based on the cross-device dragging method of the window provided in this application, during the window dragging process, the screen changes of the Windows device and the Android device can be as follows: Fig.17 As shown in (1) to (5).

[0434] In addition, it should be noted that in another possible implementation, the user may drag multiple windows displayed in the Windows device to the Android device for display. For this implementation scenario, when the floating window 50a mentioned in the above embodiment is not displayed in the Android device, when the user drags a window in the Windows device for the first time, such as the window 20a corresponding to the document 2 in the above embodiment, it can be implemented according to the processing flow of the above embodiment. In the process of dragging the window from the Windows device to the Android device for the first time, the screen changes of the Windows device and the Android device can be as follows: Fig.17 As shown in (1) to (5).

[0435] For example, when window 20a is dragged and displayed in floating window 50a of Android device, if the user opens document 1 in Windows device, the window corresponding to document 1 is as follows: Fig.18 Window 70a is shown in (1).

[0436] See also Fig.18 Zhong (1) and Fig.18In (2), illustratively, when the user operates the mouse 10a to drag the window 70a to move within the screen of the Windows device, and gradually drags it to the right edge of the screen of the Windows device, the cursor 10a will cross. Since the collaborative connection between the Windows device and the Android device is not disconnected, the floating window 50a, which is the extended screen of the Windows device, is always displayed on the screen of the Android device. Therefore, when the cursor 10a and the window 70a cross, there is no need to start the floating window 40a again, nor is there a need to re-establish the collaborative connection and the floating window 50a. Therefore, the part of the window 70a that is moved out of the Windows device (hereinafter referred to as the first part) and the cursor 10a will directly cross into the floating window 50a, and the part of the window 70a that is not moved out of the screen of the Windows device (hereinafter referred to as the second part) will be displayed on the Windows device, as shown in FIG. Fig.18 For the sake of distinction, the present embodiment shows the window 70a that passes through the floating window 50a as the window 70b.

[0437] For example, when the window 70a and the cursor 10a directly cross to the floating window 50a, while the cursor 10a drags the window 70b, the window 70b will continue to move with the cursor 10a. In this way, the first part of the window 70a displayed in the floating window 50a will become larger and larger, while the second part of the window 70a displayed in the Windows device will become smaller and smaller, until the window 70a is completely dragged to the floating window 50a for display.

[0438] In addition, it should be noted that for the window that is dragged to the floating window 50a after the window 20a, the window style displayed in the restored state (small window) on the Windows device before the window crosses will be maintained, and will be covered on the window 20b after the cross, that is, the window that crosses after the window 20a, such as the window 70b corresponding to the window 70a, will cover a part of the window 20b, such as Fig.18 As shown in (4).

[0439] In addition, it should be noted that, whether it is the window 20b or the window 70b dragged to the floating window 50a, the user can edit the content of the window displayed in the floating window 50a by using the keyboard and mouse in the keyboard and mouse sharing scenario. Moreover, the editing of the content of these windows in the floating window 50a will be synchronized to the document corresponding to the window in the Windows device. For example, after the user adds some text in the window 20a and saves the edited window 20a, the text will be added to the document 2 in the Windows device synchronously.

[0440] It should be understood that the above description is merely an example listed for a better understanding of the technical solution of this embodiment, and is not intended to be the sole limitation to this embodiment.

[0441] In addition, it is understandable that, in order to implement the above functions, the electronic device includes hardware and / or software modules corresponding to the execution of each function. In combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to be beyond the scope of the present application.

[0442] In addition, it should be noted that the cross-device dragging method of the window provided by the above embodiments implemented by the electronic device in the actual application scenario can also be performed by a chip system included in the electronic device, wherein the chip system may include a processor. The chip system can be coupled to the memory so that the chip system calls the computer program stored in the memory when it is running to implement the steps performed by the above electronic device. The processor in the chip system can be an application processor or a processor other than an application processor.

[0443] In addition, an embodiment of the present application also provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the cross-device dragging method of the window in the above-mentioned embodiment.

[0444] In addition, an embodiment of the present application further provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes the above-mentioned related steps to implement the cross-device dragging method of the window in the above-mentioned embodiment.

[0445] In addition, an embodiment of the present application also provides a chip (which may also be a component or module), which may include one or more processing circuits and one or more transceiver pins; wherein the transceiver pins and the processing circuit communicate with each other through an internal connection path, and the processing circuit executes the above-mentioned related method steps to implement the cross-device dragging method of the window in the above-mentioned embodiment, so as to control the receiving pin to receive the signal, so as to control the sending pin to send the signal.

[0446] In addition, an embodiment of the present application further provides a system, which includes the aforementioned Windows device and an Android device that establishes a mouse sharing connection with the Windows device. The Windows device and the Android device can implement dragging a window in the Windows device to the Android device for display based on the cross-device dragging method of the window provided in the above embodiment.

[0447] In addition, it can be seen from the above description that the electronic device, computer-readable storage medium, computer program product or chip provided in the embodiments of the present application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0448] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for dragging a window across devices, characterized in that: Applied to a system including a first electronic device and a second electronic device, the operating system of the first electronic device is a Windows system, and the operating system of the second electronic device is an Android system; the method includes: The first electronic device displays a first window; The first electronic device drags the first window to move within the screen of the first electronic device in response to a user operating a mouse cursor to drag the first window; When the mouse cursor moves out of the screen edge of the first electronic device, the first electronic device sends a window crossing instruction for the first window to the second electronic device, and hides the first window displayed on the first electronic device; wherein the mouse corresponding to the mouse cursor is connected to the first electronic device or the second electronic device; The second electronic device displays a first floating window on a screen of the second electronic device in response to the window crossing indication, wherein the first floating window includes a first upper status bar, a first lower status bar, and a first display area, and the first window is not displayed in the first display area; The second electronic device displays the mouse cursor in the first upper status bar and registers a mouse left button press event for the mouse; In the process of the mouse cursor dragging the first floating window to move within the screen of the second electronic device, the first electronic device sends a first indication to the second electronic device in response to a user releasing the left button of the mouse; In response to the first indication, the second electronic device registers a mouse left button lift event for the mouse to release the first floating window and display a second floating window, wherein the second floating window covers the first floating window, and the second floating window includes a second upper status bar, a second lower status bar, and a second display area; When the second electronic device receives the first collaborative screen sent by the first electronic device, the second electronic device displays the first collaborative screen in the second display area, where the first collaborative screen includes all display contents of the first window.

2. The method according to claim 1, characterized in that When the mouse cursor moves out of the screen edge of the first electronic device, the first electronic device sends a window crossing instruction for the first window to the second electronic device, and hides the first window displayed on the first electronic device, the method further includes: The second electronic device establishes a keyboard and mouse sharing connection with the first electronic device, and in the process of establishing the keyboard and mouse sharing connection, agrees on an aspect ratio of the first display area; The aspect ratio of the first display area is the same as the aspect ratio of the screen of the first electronic device.

3. The method according to claim 2, characterized in that The heights of the first upper status bar and the first lower status bar are fixed; The second electronic device displays a first floating window on a screen of the second electronic device in response to the window crossing instruction, including: The second electronic device creates a first Activity in response to the window crossing indication according to the aspect ratio, the screen size of the second electronic device, the height of the first upper status bar, the height of the first lower status bar and a preset first window proportion; and sets the window mode of the first Activity to a floating window mode to obtain the first floating window.

4. The method according to claim 3, characterized in that The second electronic device creates a first Activity in response to the window crossing indication according to the aspect ratio, the screen size of the second electronic device, the height of the first upper status bar, the height of the first lower status bar, and a preset first window proportion, including: The second electronic device determines, according to the aspect ratio, the screen size of the second electronic device and a preset first window proportion, a width of the first display area and a height of the first display area; The second electronic device obtains the Y coordinate of the first window crossing carried in the window crossing indication, and uses the Y coordinate of the first window crossing as the starting Y coordinate of the height of the first Activity; The second electronic device determines, according to the starting Y coordinate, the height of the first upper status bar, the height of the first display area, and the height of the first lower status bar, the ending Y coordinate of the height of the first Activity; The second electronic device sets a first bounds attribute of the first Activity according to the width of the first display area, the start Y coordinate, and the end Y coordinate; The second electronic device creates the first Activity according to the first bounds attribute.

5. The method according to claim 4, characterized in that Display the second floating window, including: The second electronic device creates a second Activity according to the first Activity; and sets the window mode of the second Activity to a floating window mode to obtain the second floating window.

6. The method according to claim 5, characterized in that After displaying the second floating window, the method further includes: The second electronic device destroys the first floating window.

7. The method according to claim 5, characterized in that The second upper status bar has the same height as the first upper status bar, and the second lower status bar has the same height as the first lower status bar; The second electronic device creates a second Activity according to the first Activity, including: The second electronic device obtains a first bounds attribute of the first Activity; When the first bounds attribute is acquired, the second electronic device sets a second bounds attribute of the second Activity according to the first bounds attribute; The second electronic device creates the second Activity according to the second bounds attribute.

8. The method according to claim 7, characterized in that The second bounds attribute is the same as the first bounds attribute.

9. The method according to claim 7, characterized in that: The method further comprises: When the first bounds attribute is not obtained, the second electronic device determines the width of the second display area and the height of the second display area according to the aspect ratio, the screen size of the second electronic device and the preset first window proportion; The second electronic device obtains the Y coordinate of the first window crossing carried in the window crossing indication, and uses the Y coordinate of the first window crossing as the starting Y coordinate of the height of the second Activity; The second electronic device determines, according to the starting Y coordinate, the height of the second upper status bar, the height of the second display area, and the height of the second lower status bar, the ending Y coordinate of the height of the second Activity; The second electronic device sets the second bounds attribute of the second Activity according to the width of the second display area, the starting Y coordinate, and the ending Y coordinate.

10. The method according to claim 3, characterized in that: Create the first Activity, also include: The window attribute of the first Activity is set to disable scaling mode.

11. The method according to any one of claims 1 to 10, characterized in that: The second electronic device displays the mouse cursor in the first upper status bar, comprising: The second electronic device selects a coordinate point from the currently displayed first upper status bar as the initial mouse coordinate according to the display position of the first floating window; wherein the initial mouse coordinate avoids the area where the function keys included in the first upper status bar are located and the hot zone of the side edge of the screen of the second electronic device; The second electronic device displays the mouse cursor at the initial coordinates of the mouse.

12. The method according to any one of claims 1 to 11, characterized in that: The first indication includes a release window indication and a screen coordination indication; The release window instruction is used to instruct the second electronic device to release the first floating window; The screen cooperation instruction is used to instruct the second electronic device to establish a screen cooperation connection with the first electronic device in which the cooperation mode is an extended mode; Wherein, the second floating window is displayed on the screen of the second electronic device after the screen cooperative connection is successfully established.

13. The method according to claim 12, characterized in that The method further comprises: The second electronic device registers a drag monitor for the first upper status bar; The second electronic device is further configured to cancel the drag monitoring in response to the release window indication.

14. The method according to claim 13, characterized in that Before canceling the drag monitoring, the method further includes: When the drag monitoring is triggered and the window movement position information sent by the first electronic device is received, the second electronic device moves the display position of the first floating window according to the window movement position information.

15. The method according to any one of claims 1 to 14, characterized in that The second floating window is displayed in the second electronic device whose screen state is in a horizontal state; After displaying the first collaborative screen in the second display area, the method further includes: In response to the screen rotation operation, the second electronic device displays a third floating window at a preset position from the upper edge of the screen in the portrait state, wherein the third floating window includes a third upper status bar, a third lower status bar, and a third display area; The second electronic device displays the first collaborative screen in the third display area; and destroys or hides the second Activity; Among them, the heights of the third upper status bar and the third lower status bar are fixed, the aspect ratio of the third display area is the same as the aspect ratio of the first display area agreed with the first electronic device when establishing the keyboard and mouse sharing connection, and the aspect ratio of the first display area is the same as the aspect ratio of the screen of the first electronic device.

16. The method according to claim 15, characterized in that The method further comprises: The second electronic device scales the second floating window in response to a scaling operation on the second floating window; Among them, the aspect ratio of the second display area of ​​the second floating window after being enlarged or reduced is the same as the aspect ratio of the first display area agreed with the first electronic device when establishing the keyboard and mouse sharing connection, and the aspect ratio of the first display area is the same as the aspect ratio of the screen of the first electronic device.

17. The method according to any one of claims 1 to 16, characterized in that: After displaying the first collaborative screen in the second display area, the method further includes: The first electronic device displays a second window; The first electronic device drags the second window to move within the screen of the first electronic device in response to a user operating a mouse cursor to drag the second window; When the mouse cursor moves out of the screen edge of the first electronic device, the first electronic device sends a second collaborative screen for the second window to the second electronic device, the second collaborative screen includes a first part of the second window, and displays the second part of the second window on the screen of the first electronic device; wherein the first part is the part that moves out of the screen edge of the first electronic device, and the second part is the part that does not move out of the screen edge of the first electronic device; When the second electronic device receives the second collaborative screen sent by the first electronic device, the second electronic device displays the second collaborative screen in the second display area; wherein the window state of the second window in the second collaborative screen is a restored state, and the second collaborative screen blocks the first collaborative screen; Wherein, when the window state is the restored state, the title bar of the first window includes a maximize control but does not include a restore control.

18. The method according to any one of claims 1 to 17, characterized in that When the mouse cursor moves out of the screen edge of the first electronic device, the first electronic device sends a window crossing instruction for the first window to the second electronic device, and hides the first window displayed on the first electronic device, including: When the mouse cursor moves out of the screen edge of the first electronic device, the first electronic device sends a window crossing instruction for the first window to the second electronic device; After sending the window crossing instruction to the second electronic device, the first electronic device hides the first window displayed on the first electronic device when receiving a window crossing confirmation message made by the second electronic device in response to the window crossing instruction.

19. The method according to claim 18, characterized in that After the second electronic device displays the first floating window on the screen of the second electronic device in response to the window crossing instruction, the method further includes: The second electronic device generates the window crossing confirmation message for the window crossing indication, and sends the window crossing message to the first electronic device.

20. The method according to any one of claims 1 to 19, characterized in that After the second electronic device registers a left mouse button lift event for the mouse in response to the first indication to release the first floating window and display a second floating window, the method further includes: The second electronic device generates a screen coordination connection success message for the screen coordination indication, and sends the screen coordination connection success message to the first electronic device.

21. The method according to claim 20, characterized in that The first collaborative screen also includes a task bar of the first electronic device; The method further comprises: In response to the screen cooperative connection success message, the first electronic device sets the window state of the first window to a maximized state so that the first window displays all display contents included therein; The first electronic device synthesizes and draws the first collaborative screen based on the first window in the maximized state and the task bar of the first electronic device; and sends the obtained first collaborative screen to the second electronic device.

22. The method according to claim 21, characterized in that When the window state is the maximized state, the title bar of the first window includes a restore control but does not include a maximize control.

23. The method according to any one of claims 1 to 22, characterized in that The second electronic device includes a screen computer and a foldable screen mobile phone.

24. A window cross-device dragging system, characterized in that: include: A first electronic device and a second electronic device, a keyboard and mouse sharing connection is established between the first electronic device and the second electronic device, the operating system of the first electronic device is a Windows system, and the operating system of the second electronic device is an Android system; The first electronic device and the second electronic device are used to execute the cross-device dragging method of a window as described in any one of claims 1 to 23.

25. An electronic device, characterized in that: The electronic device includes: a memory and a processor, the memory and the processor are coupled; the memory stores program instructions, and when the program instructions are executed by the processor, the electronic device executes the cross-device dragging method of the window as described in any one of claims 1 to 23.

26. A computer-readable storage medium, characterized in that: It comprises a computer program, which, when executed on an electronic device, enables the electronic device to execute the cross-device dragging method of a window as described in any one of claims 1 to 23.

Images