Cross-device cooperation method and electronic device

Abstract

The invention provides a cross-device cooperation method and an electronic device, a user can drag an operation object on a starting device to a trigger area on the starting device, then cross-device dragging of the operation object can be triggered, and a target device can correspondingly display the operation object. A user does not need to trigger cross-device dragging by continuously dragging the operation object from the user interface of the starting device to the user interface of the target device, the influence of the hardware condition of the starting device on cross-device dragging between the starting device and the target device is small, and the efficiency of cross-device dragging between the starting device and the target device is higher. The cooperative task processing capability between the two devices is higher, and the user experience is better.

Description

Technical Field

[0001] This application relates to the field of terminal device software. Specifically, it relates to a method and an electronic device for cross-device collaboration. Background Art

[0002] With the development of electronic technology and mobile Internet, the same user often has multiple electronic devices at the same time. For example, mobile phones, tablets, personal computers, smart home devices, etc. There are also more and more scenarios that require multiple electronic devices to be connected and cooperate with each other, such as collaborative office and smart home.

[0003] Generally, different devices can have different capabilities. For example, communication capabilities, data processing capabilities, and image processing capabilities. Compared with multiple devices functioning independently, the collaborative work between multiple devices can make more full use of the different capabilities of different types of electronic devices, improve the processing capabilities of multiple devices for different tasks in different scenarios, and enhance the user experience.

[0004] How to improve the efficiency of collaborative task processing between multiple devices is a problem worthy of consideration. Summary of the Invention

[0005] This application provides a method for cross-device collaboration. The user can drag an operation object located on a first device to a trigger area of the first device to trigger the cross-device drag of the operation object between the first device and a second device, and the cross-device drag of the operation object between the two devices can be realized without a long drag stroke. The efficiency of data sharing and capability sharing between the first device and the second device is higher, and the ability of the two devices to collaborate on tasks is stronger.

[0006] In a first aspect, a method for cross-device collaboration is provided, including: the first device displays a start interface, and the start interface includes an operation object; in response to the operation of dragging the operation object to the trigger area of the first device and releasing it, the second device displays the operation object, and the trigger area is located on the outer periphery of the display area of the first device; wherein, the first device and the second device are connected to each other, and the second device determines according to one or more of the following: the relative position relationship between the second device and the first device, the placement posture of the second device, or the direction in which the operation object is dragged.

[0007] In a possible implementation manner, the first device and the second device can be communicatively connected by wireless means and / or wired means.

[0008] In a possible implementation, the operation object can be one or more interface elements on the start interface or the content of the interface elements. Exemplarily, the operation object can include one or more of the following: text, pictures, code, files, interface components, and application windows, etc. Among them, the interface components can include buttons, text boxes, sliders, and icons, etc. The application windows can include full-screen windows, split-screen windows, or floating windows, etc.

[0009] In some scenarios, the trigger area can also be referred to as a drag perception area or a cross-device drag operation perception area.

[0010] In a possible implementation, the trigger area can include multiple sub-areas on the outer periphery of the display area of the first device.

[0011] In this technical solution, the first device can be provided with a perception area for triggering cross-device dragging. The user can drag the operation object on the first device to this perception area to trigger the cross-device dragging of the operation object, without continuously dragging the operation object from the first device to the second device. The efficiency of cross-device dragging between the first device and the second device is higher, which is beneficial to improving the efficiency of collaborative task processing between the two devices.

[0012] In combination with the first aspect, in certain implementations of the first aspect, in response to the operation of dragging the operation object to the trigger area of the first device and releasing it, the second device displays the operation object, including: the second device displays the operation object in the response area, and the response area is located on the outer periphery of the display area of the second device.

[0013] In a possible implementation, the response area is located on the edge of the display area of the second device on the side close to the first device.

[0014] The operation object released from the outer periphery of the display area of the first device appears on the outer periphery of the display area of the second device in the case of triggering cross-device dragging. This technical solution is more in line with the user's intuition of performing cross-device dragging. On the one hand, it is beneficial to improving the efficiency of cross-device dragging between the two devices. On the other hand, compared with displaying the operation object in the middle area of the display area of the second device, this solution is beneficial to reducing the interference of cross-device dragging on the tasks being executed on the second device and is beneficial to improving the user experience.

[0015] In combination with the first aspect, in certain implementations of the first aspect, the second device displays the operation object in the response area, including: the second device displays the operation object at the exit position of the response area, and the exit position is determined according to the entry position, and the entry position is the position where the operation object is released in the trigger area.

[0016] In a possible implementation, the distances from the exit position to the upper edge and the lower edge of the second device are L1 and L2 respectively, and the distances from the entrance position to the upper edge and the lower edge of the first device are L3 and L4 respectively. Then: L1:L2 = L3:L4.

[0017] In a possible implementation, the first device sends the information of the entrance position to the second device.

[0018] Using the entrance position where the trigger operation occurs during the cross-device drag to determine the exit position where the operation object appears on the second device. This technical solution is more in line with the user's intuition of performing cross-device drag, which is beneficial to improving the efficiency of cross-device drag between the two devices and enhancing the user experience.

[0019] Combined with the first aspect, in some implementations of the first aspect, one or more shortcuts are also displayed in the response area. The shortcuts are used to indicate the display mode of the operation object. The method further includes: in response to the operation of dragging the operation object to the display area of the second shortcut, the second device displays the operation object in a second display mode. The second shortcut is used to indicate the second display mode, and one or more shortcuts include the second shortcut.

[0020] In some scenarios, the shortcuts displayed in the response area can also be referred to as the indication information of the application mode of the second device.

[0021] In some scenarios, the display mode of the operation object can also be understood as the application mode of the second device.

[0022] In a possible implementation, the application modes of the second device include: mirroring, stretching, sharing, etc. The second application mode can be any one of these application modes.

[0023] During each cross-device drag operation, the user can reselect the application mode of the second device, which is to a certain extent beneficial to improving the efficiency of collaborative work between the first device and the second device and enhancing the user experience of cross-device drag operations. Combined with the first aspect, in some implementations of the first aspect, the second device displays the operation object in the response area, including: the second device highlights the operation object in the response area.

[0024] In a possible implementation, the second device can display the operation object in a highlighted color. The second device can also display the operation object with animation effects (such as jittering, deformation, etc.). The second device can also display the operation object in an enlarged manner. The second device can also display the operation object by hovering at the position in the response area.

[0025] By highlighting the operation object of cross-device dragging to remind the user to process it in time, the implementation of this technical solution is beneficial to improving the success rate of cross-device dragging of the operation object, and to a certain extent, is beneficial to improving the efficiency of collaborative work between the first device and the second device, and enhancing the user experience of cross-device dragging operations.

[0026] Combined with the first aspect, in some implementation manners of the first aspect, the method further includes: when the operation object is displayed on the second device,

[0027] In response to the operation object not being operated within a preset time period, the second device stops displaying the operation object.

[0028] In some scenarios, the above solution can also be understood as: in response to the user not operating the operation object within a preset time period, the second device can return to the state before performing the cross-device dragging operation.

[0029] In this technical solution, a time threshold is set to control the user's response to the operation object of the cross-device dragging operation. When the time threshold is exceeded, the first device and the second device can stop executing the cross-device dragging task. The implementation of this technical solution is beneficial to reducing the impact of cross-device dragging on other tasks that the first device and the second device are performing or about to perform, and is beneficial to improving the utilization efficiency of the software resources and hardware resources of the first device and the second device.

[0030] Combined with the first aspect, in some implementation manners of the first aspect, when the operation object is an application window, in response to the operation of dragging the operation object to the trigger area of the first device and releasing it, the second device displays the operation object, including: the second device displays the application window in a preset manner, and the preset manner is determined according to the type of the second device. The preset manner includes any one of a floating window, a split-screen window, a multi-window, or a full-screen window.

[0031] For the case where the operation object is an application window, the second device can determine the display manner of the operation object of the cross-device dragging operation according to the size of its own display area, which is beneficial to improving the display effect of the operation object on the second device, beneficial to enhancing the user experience of cross-device dragging operations, and to a certain extent, also beneficial to enhancing the collaborative work ability between the first device and the second device.

[0032] Combined with the first aspect, in some implementation manners of the first aspect, in response to the operation of dragging the operation object to the trigger area of the first device and releasing it, displaying the operation object on the second device includes: the second device displays the user interface of the target application, and the target application is used to edit the operation object.

[0033] In response to a cross-device drag operation by the user, the second device can directly open the application window for processing the operation object and display the user interface for editing or processing the operation object. The implementation of this technical solution is beneficial to improving the efficiency of performing cross-device drag tasks between the two devices, enhancing the user experience, and to a certain extent, also beneficial to improving the collaborative working ability between the first device and the second device.

[0034] In combination with the first aspect, in some implementation manners of the first aspect, one or more shortcuts are also displayed in the triggering area. The shortcuts are used to indicate the display manner of the operation object. In response to the operation of dragging the operation object to the triggering area of the first device and releasing it, the second device displays the operation object, including: in response to the operation of dragging the operation object to the display area of the first shortcut, the second device displays the operation object in the first display manner. The first shortcut is used to indicate the first display manner, and the one or more shortcuts include the first shortcut.

[0035] In some scenarios, the shortcuts displayed in the triggering area can also be referred to as the indication information of the application manner of the second device.

[0036] In some scenarios, the display manner of the operation object can also be understood as the application manner of the second device.

[0037] In a possible implementation manner, the application manner of the second device includes: mirroring, stretching, sharing, etc., and the second application manner can be any one of these application manners.

[0038] During each cross-device drag operation, the user can re-select the application manner of the second device, which is beneficial to improving the efficiency of collaborative work between the first device and the second device to a certain extent and enhancing the user experience of the cross-device drag operation.

[0039] In combination with the first aspect, in some implementation manners of the first aspect, in response to the operation of dragging the operation object to the triggering area of the first device and releasing it, the second device displays the operation object, including:

[0040] In response to the operation of using the first operation body to act on the first object in the first operation manner, the second device displays the target interface including the first object, or in response to the operation of using the second operation body to act on the second object in the second operation manner, the second device displays the target interface including the second operation object; wherein, the operation object includes the first object and the second object, the first object is different from the second object, the first operation body is different from the second operation body, and / or, the first operation manner is different from the second operation manner.

[0041] In a possible implementation, in response to an operation acting on a first object using a first operation method, a part of the first object is selected as the operation object; in response to an operation acting on the first object using a second operation method, the first object is selected as the operation object; wherein, the first operation method is different from the second operation method.

[0042] In a possible implementation, the first operation body or the second operation body may include a finger, a stylus, a mouse, etc. The first operation method or the second operation method may include a single-finger operation, a two-finger operation, a palm operation, a long-press operation, a hard-press operation, a knuckle operation, etc.

[0043] By using different operation bodies and / or operation methods to trigger cross-device dragging of different types of operation objects, the implementation of this technical solution is beneficial to improving the applicability of the cross-device dragging method provided in this application in different scenarios, beneficial to improving the efficiency of multi-device execution of cross-device dragging tasks, and also beneficial to improving the collaborative working ability between the first device and the second device.

[0044] In combination with the first aspect, in some implementations of the first aspect, before the second device displays the operation object in response to the operation of dragging the operation object to the trigger area of the first device and releasing it, the method further includes: in response to the distance between the operation object and the edge of the display area of the first device being less than a first threshold, the first device displays an indication control, and the indication control is used to indicate the trigger area.

[0045] In this technical solution, the indication control is used to indicate the recognition area for triggering cross-device dragging. The position of the recognition area is more easily perceived by the user, the success rate of the user's cross-device dragging operation is higher, and the user's experience of the cross-device dragging operation is better.

[0046] In combination with the first aspect, in some implementations of the first aspect, the trigger area is located at the edge of the display area on the side of the first device close to the second device.

[0047] Setting the trigger area at the position of the edge of the display area of the first device close to the second device, this technical solution is more in line with the user's intuition for performing cross-device dragging, beneficial to improving the efficiency of cross-device dragging between the two devices, and beneficial to improving the user experience.

[0048] In combination with the first aspect, in some implementations of the first aspect, before the second device displays the operation object in response to the operation of dragging the operation object to the trigger area of the first device and releasing it, the method further includes: the first device displays the identification information of the second device in the trigger area; in response to the user dragging the operation object to the display area of the identification information of the second device, the second device displays the operation object.

[0049] This technical solution combines multiple factors to determine the second device for cross-device drag operations, which helps to make the selected second device more suitable for performing cross-device drag operations and more suitable for collaborating with the first device. The implementation of this technical solution helps to improve the collaborative working ability between the first device and the second device.

[0050] For the relevant explanations and descriptions of the beneficial effects in the following technical solutions, reference can be made to the relevant content in the first aspect above, which will not be elaborated here.

[0051] In a second aspect, a cross-device collaboration method is provided, which is applied to a first device and includes: displaying a start interface, where the start interface includes an operation object; in response to an operation of dragging the operation object to a trigger area and releasing it, sending the operation object to a second device, where the trigger area is located outside the display area of the first device; where the first device and the second device are connected to each other, and the second device determines based on one or more of the following: the relative position relationship between the second device and the first device, the placement posture of the second device, or the direction in which the operation object is dragged.

[0052] In combination with the second aspect, in some implementation manners of the second aspect, before sending the operation object to the second device in response to an operation of dragging the operation object to the trigger area and releasing it, one or more shortcuts are displayed in the trigger area, and the shortcuts are used to indicate the display manner of the operation object; in response to an operation of dragging the operation object to the display area of the first shortcut, an indication message is sent to the second device, and the indication message is used to indicate the first display manner, the first shortcut is used to indicate the first display manner, and the one or more shortcuts include the first shortcut.

[0053] In combination with the second aspect, in some implementation manners of the second aspect, sending the operation object to the second device in response to an operation of dragging the operation object to the trigger area and releasing it includes: displaying the identification information of the second device in the trigger area; in response to an operation of dragging the operation object to the display area of the identification information of the second device and releasing it, sending the operation object to the second device.

[0054] In combination with the second aspect, in some implementation manners of the second aspect, before sending the operation object to the second device in response to an operation of dragging the operation object to the trigger area and releasing it, in response to an operation of using a first operation manner on a first object, a part of the first object is selected as the operation object; in response to an operation of using a second operation manner on the first object, the first object is selected as the operation object; where the first operation manner is different from the second operation manner.

[0055] In combination with the second aspect, in certain implementations of the second aspect, in response to the operation of dragging the operation object to the trigger area and releasing it, before sending the operation object to the second device, in response to the distance between the operation object and the edge of the display area of the first device being less than a first threshold, an indicator control is displayed, and the indicator control is used to indicate the trigger area.

[0056] In combination with the second aspect, in some implementations of the second aspect, the trigger area is located at an edge of a display area of the first device on a side close to the second device.

[0057] In a third aspect, a method for cross-device coordination is provided, which is applied to a first device, including: displaying a starting interface, the starting interface including an operation object; in response to a first drag operation on the operation object along a first drag direction, displaying a first trigger area on the periphery of the starting interface corresponding to the first drag direction; in response to a release operation on the operation object in the first trigger area, sending the operation object to a second device connected to the first device.

[0058] In a possible implementation, a connection direction between the second device and the first device is parallel to the dragging direction.

[0059] It should be understood that the “parallel” here can be understood as approximately parallel or substantially parallel, etc.

[0060] In combination with the third aspect, in certain implementations of the third aspect, the method also includes: in response to a second drag operation on the operation object along the second drag direction, displaying a second trigger area on the periphery of the starting interface corresponding to the second drag direction; in response to a release operation on the operation object in the second trigger area, sending the operation object to a third device connected to the first device.

[0061] In combination with the third aspect, in certain implementations of the third aspect, the method also includes: in response to the operation of dragging the operation object to the first trigger area, displaying one or more device identifiers in the first trigger area, the one or more device identifiers are used to indicate candidate devices, the candidate devices are located on the side of the first device close to the first trigger area, and the candidate devices include the second device; in response to the release operation of the operation object in the first trigger area, sending the operation object to the second device connected to the first device, including: in response to the operation of dragging the operation object to the display area of the device identifier of the second device and releasing it, sending the operation object to the second device.

[0062] In combination with the third aspect, in certain implementations of the third aspect, there are multiple device identifiers, and the relative positions of the multiple device identifiers are the same as the relative positions of the multiple candidate devices corresponding to the multiple device identifiers.

[0063] In combination with the third aspect, in certain implementations of the third aspect, the device identifier is further used to indicate the placement posture of the candidate device.

[0064] In a fourth aspect, a device for cross-device collaboration is provided. The device includes a first module and a second module. The first module is configured to: display a start interface, where the start interface includes an operation object; The second module is configured to: in response to an operation of dragging the operation object to a trigger area of the first module and releasing it, display the operation object, where the trigger area is located on the outer periphery of the display area of the first module; Wherein, the first module and the second module are interconnected, and the second module determines based on one or more of the following: the relative position relationship between the second module and the first module, the placement posture of the second module, or the direction in which the operation object is dragged.

[0065] In combination with the fourth aspect, in some implementation manners of the fourth aspect, the second module is specifically configured to: display the operation object in a response area, where the response area is located on the outer periphery of the display area of the second module.

[0066] In combination with the fourth aspect, in some implementation manners of the fourth aspect, the second module is specifically configured to: display the operation object at an exit position of the response area, where the exit position is determined according to an entry position, and the entry position is the position where the operation object is released in the trigger area.

[0067] In combination with the fourth aspect, in some implementation manners of the fourth aspect, one or more shortcuts are also displayed in the response area, where the shortcuts are used to indicate the display manner of the operation object, and the second module is further configured to: in response to an operation of dragging the operation object to the display area of the second shortcut, display the operation object in a second display manner, where the second shortcut is used to indicate the second display manner, and the one or more shortcuts include the second shortcut.

[0068] In combination with the fourth aspect, in some implementation manners of the fourth aspect, the second module is further configured to: in the case where the second module displays the operation object, stop displaying the operation object in response to the operation object not being operated within a preset duration.

[0069] In combination with the fourth aspect, in some implementation manners of the fourth aspect, the operation object is an application window, and the second module is specifically configured to: display the application window in a preset manner, where the preset manner is determined according to the type of the second module, and the preset manner includes any one of a floating window, a split-screen window, a multi-window, or a full-screen window.

[0070] In combination with the fourth aspect, in some implementation manners of the fourth aspect, the second module is specifically configured to: display the user interface of a target application, where the target application is used to edit the operation object.

[0071] In combination with the fourth aspect, in some implementations of the fourth aspect, one or more shortcuts are also displayed in the trigger area, and the shortcuts are used to indicate the display mode of the operation object. The second module is specifically configured to: in response to an operation of dragging the operation object to the display area of the first shortcut, display the operation object in the first display mode, where the first shortcut is used to indicate the first display mode, and the one or more shortcuts include the first shortcut.

[0072] In combination with the fourth aspect, in some implementations of the fourth aspect, the second module is specifically configured to: in response to an operation of using the first operation body to act on the first object in the first operation mode, display a target interface including the first object, or in response to an operation of using the second operation body to act on the second object in the second operation mode, display a target interface including the second operation object; where the operation object includes the first object and the second object, the first object is different from the second object, the first operation body is different from the second operation body, and / or the first operation mode is different from the second operation mode.

[0073] In combination with the fourth aspect, in some implementations of the fourth aspect, the first module is further configured to: in response to the distance between the operation object and the edge of the display area of the first module being less than a first threshold, display an indication control, where the indication control is used to indicate the trigger area.

[0074] In combination with the fourth aspect, in some implementations of the fourth aspect, the trigger area is located at the edge of the display area on the side of the first module close to the second module.

[0075] In combination with the fourth aspect, in some implementations of the fourth aspect, before the second module displays the operation object in response to an operation of dragging the operation object to the trigger area of the first module and releasing it, the first module is configured to: display the identification information of the second device in the trigger area; in response to the user dragging the operation object to the display area of the identification information of the second module, the second module is configured to: display the operation object.

[0076] Fifth aspect, a first device is provided, the first device includes a processor and a memory, the memory is used to store program instructions, and the processor is configured to: display a start interface, the start interface includes an operation object; in response to an operation of dragging the operation object to the trigger area and releasing it, send the operation object to a second device, where the trigger area is located on the outer periphery of the display area of the first device; where the first device and the second device are interconnected, and the second device determines according to one or more of the following: the relative position relationship between the second device and the first device, the placement posture of the second device, or the direction in which the operation object is dragged..

[0077] In combination with the fifth aspect, in some implementations of the fifth aspect, the processor is further configured to: in response to the distance between the operation object and the edge of the display area of the first device being less than a first threshold, display an indication control, where the indication control is used to indicate the trigger area.

[0078] In combination with the fifth aspect, in some implementations of the fifth aspect, the trigger area is located at the edge of the display area on the side of the first device close to the second device.

[0079] In combination with the fifth aspect, in some implementations of the fifth aspect, one or more shortcuts are also displayed in the trigger area, where the shortcuts are used to indicate the display mode of the operation object.

[0080] In a possible implementation, the processor is further configured to: send first indication information, where the first indication information is used to indicate the display mode of the operation object.

[0081] In combination with the fifth aspect, in some implementations of the fifth aspect, the processor is further configured to: send second indication information, where the second indication information is used to indicate the entry position, and the entry position is the position where the operation object is released in the trigger area.

[0082] In combination with the fifth aspect, in some implementations of the fifth aspect, in response to the operation of dragging the operation object to the trigger area of the first device and releasing it, the processor is further configured to: display the identification information of the second device in the trigger area.

[0083] Sixth aspect, there is provided a second device, where the second device includes a processor and a memory, the memory is used to store program instructions, and the processor is used to: receive an operation object and display the operation object.

[0084] In combination with the sixth aspect, in some implementations of the sixth aspect, the processor is specifically configured to: display the operation object in a response area, where the response area is located on the outer periphery of the display area of the second device.

[0085] In combination with the sixth aspect, in some implementations of the sixth aspect, the processor is specifically configured to: display the operation object at the exit position of the response area, where the exit position is determined according to the entry position, and the entry position is the position where the operation object is released in the trigger area.

[0086] In combination with the sixth aspect, in some implementations of the sixth aspect, one or more shortcuts are also displayed in the response area, where the shortcuts are used to indicate the display mode of the operation object, and the processor is further configured to: in response to the operation of dragging the operation object to the display area of the second shortcut, display the operation object in a second display mode, where the second shortcut is used to indicate the second display mode, and the one or more shortcuts include the second shortcut.

[0087] In combination with the sixth aspect, in some implementations of the sixth aspect, the processor is further configured to: when the second device displays an operation object, in response to the operation object not being operated within a preset duration, stop displaying the operation object.

[0088] In combination with the sixth aspect, in some implementations of the sixth aspect, the operation object is an application window, and the processor is specifically configured to: display the application window in a preset manner, where the preset manner is determined according to the type of the second device, and the preset manner includes any one of a floating window, a split-screen window, a multi-window, or a full-screen window.

[0089] In combination with the sixth aspect, in some implementations of the sixth aspect, the processor is specifically configured to: display the user interface of the target application, where the target application is used to edit the operation object.

[0090] In combination with the sixth aspect, in some implementations of the sixth aspect, the processor is specifically configured to: receive first indication information, where the first indication information is used to indicate the display manner of the operation object. The processor is further configured to: display the operation object in a first display manner according to the first indication information, where the first shortcut is used to indicate the first display manner, and one or more shortcuts include the first shortcut.

[0091] In combination with the sixth aspect, in some implementations of the sixth aspect, the processor is specifically configured to: in response to an operation of using a first operation body to act on a first object in a first operation manner, display a target interface including the first object, or, in response to an operation of using a second operation body to act on a second object in a second operation manner, display a target interface including the second operation object; where the operation object includes the first object and the second object, the first object is different from the second object, the first operation body is different from the second operation body, and / or, the first operation manner is different from the second operation manner.

[0092] In a seventh aspect, there is provided a computer program product, which includes computer program code. When the computer program code runs on a computer, it causes the methods in the second aspect and any possible implementations thereof to be executed, or causes the methods in the third aspect and any possible implementations thereof to be executed.

[0093] In an eighth aspect, there is provided a computer-readable storage medium, which stores computer program code. When the computer program code runs on a computer, it causes the methods in the second aspect and any possible implementations thereof to be executed, or causes the methods in the third aspect and any possible implementations thereof to be executed.

[0094] In a ninth aspect, a chip is provided, including a processor configured to read instructions stored in a memory. When the processor executes the instructions, the chip implements the methods in the second aspect and any possible implementation manners thereof, or the chip implements the methods in the third aspect and any possible implementation manners thereof. Description of the Drawings

[0095] Figure 1 FIG. is a schematic diagram of a hardware architecture of an electronic device applicable to an embodiment of the present application.

[0096] Figure 2 FIG. is a schematic diagram of a software architecture of an electronic device applicable to an embodiment of the present application.

[0097] Figure 3 FIG. is a schematic diagram of a process of a cross-device drag-and-drop operation provided by an embodiment of the present application.

[0098] Figure 4 FIG. is a schematic diagram of a process of another cross-device drag-and-drop operation provided by an embodiment of the present application.

[0099] Figure 5 FIG. is a schematic diagram of a drag sensing area of a starting device provided by an embodiment of the present application.

[0100] Figure 6 FIG. is a schematic diagram of a method for selecting a target device provided by an embodiment of the present application.

[0101] Figure 7 FIG. is a schematic diagram of another method for selecting a target device provided by an embodiment of the present application.

[0102] Figure 8 FIG. is a schematic diagram of yet another method for selecting a target device provided by an embodiment of the present application.

[0103] Figure 9 FIG. is a schematic diagram of yet another method for selecting a target device provided by an embodiment of the present application.

[0104] Figure 10 FIG. is a schematic diagram of a method for selecting an operation object provided by an embodiment of the present application.

[0105] Figure 11 FIG. is a schematic diagram of a method for selecting an application mode of a target device provided by an embodiment of the present application.

[0106] Figure 12 FIG. is a schematic diagram of a process of yet another cross-device drag-and-drop operation provided by an embodiment of the present application.

[0107] Figures 13 to 18 FIG. is a schematic diagram of a method for a target device to process an operation object of a cross-device drag-and-drop operation provided by an embodiment of the present application.

[0108] Figure 19 It is a schematic diagram of a method for collaborative work between devices provided by an embodiment of the present application.

[0109] Figure 20 It is a schematic diagram of a cross-device collaboration device provided by an embodiment of the present application.

[0110] Figure 21 It is a schematic diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0111] Next, the technical solutions in the present application will be described in conjunction with the accompanying drawings.

[0112] The terms used in the following embodiments are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification and appended claims of the present application, the singular forms "a", "an", "the", "above-mentioned", "said", and "this" are also intended to include expressions such as "one or more", unless there is a clear contrary indication in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" means one, two or more than two. The term "and / or" is used to describe the association relationship of associated objects and means that three relationships can exist; for example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone, where A and B can be singular or plural. The character " / " generally means that the associated objects before and after are in an "or" relationship.

[0113] Referring to "one embodiment" or "some embodiments" described in this specification means that specific features, structures, or characteristics described in conjunction with the embodiment are included in one or more embodiments of the present application. Thus, the statements "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "include", "comprise", "have" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0114] Figure 1A schematic structural diagram of the electronic device 100 is shown. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0115] It can be understood that the structure schematically shown in the embodiments of this application does not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than those shown in the figure, or combine certain components, or split certain components, or have different component arrangements. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0116] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0117] Among them, the controller may be the nerve center and command center of the electronic device 100. The controller may generate operation control signals according to the instruction operation code and timing signal to complete the control of fetching instructions and executing instructions.

[0118] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0119] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0120] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display 194 and the camera 193. MIPI interfaces include the camera serial interface (CSI) and the display serial interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the camera function of the electronic device 100. The processor 110 and the display 194 communicate via the DSI interface to implement the display function of the electronic device 100.

[0121] The USB interface 130 is an interface that complies with USB standards and may be a Mini USB interface, a Micro USB interface, a USB Type-C interface, or the like. The USB interface 130 can be used to connect a charger to charge the electronic device 100, or to transfer data between the electronic device 100 and peripheral devices. It can also be used to connect headphones to play audio. This interface can also be used to connect other electronic devices, such as augmented reality devices.

[0122] It can be understood that the interface connection relationships between the various modules illustrated in the embodiments of the present application are only illustrative descriptions and do not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods or a combination of multiple interface connection methods in the above embodiments.

[0123] The wireless communication function of the electronic device 100 can be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modulation and demodulation processor, and the baseband processor, etc.

[0124] The electronic device 100 realizes the display function through the GPU, the display screen 194, and the application processor, etc. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to execute mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change the display information.

[0125] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel may adopt a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light emitting diode or an active-matrix organic light emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0126] The electronic device 100 can realize the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194, and the application processor, etc. The ISP is used to process the data fed back by the camera 193. The camera 193 is used to capture still images or videos. The digital signal processor is used to process digital signals, and in addition to processing digital image signals, it can also process other digital signals. The video codec is used to compress or decompress digital videos.

[0127] The NPU is a neural-network (NN) computing processor. By learning from the structure of biological neural networks, such as the transmission pattern between human brain neurons, it can quickly process input information and can also continuously self-learn.

[0128] The external memory interface 120 can be used to connect to an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to achieve the data storage function.

[0129] The internal memory 121 can be used to store computer-executable program code, and the executable program code includes instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 can include a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.). The data storage area can store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.). In addition, the internal memory 121 can include a high-speed random access memory and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0130] The electronic device 100 can implement audio functions through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone interface 170D, and the application processor, etc. Such as music playback, recording, etc.

[0131] The keys 190 include a power-on key, volume keys, etc. The keys 190 can be mechanical keys or touch keys. The electronic device 100 can receive key inputs and generate key signal inputs related to the user settings and function controls of the electronic device 100.

[0132] The motor 191 can generate a vibration prompt. The motor 191 can be used for incoming call vibration prompts and can also be used for touch vibration feedback.

[0133] The indicator 192 can be an indicator light and can be used to indicate the charging state, the change in battery power, and can also be used to indicate messages, missed calls, notifications, etc.

[0134] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservices architecture, or a cloud architecture. In the embodiments of this application, the Android system with a layered architecture is taken as an example to exemplarily illustrate the software structure of the electronic device 100.

[0135] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0136] Figure 2 This is a block diagram of the software structure of the electronic device 100 according to an embodiment of the present application. The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers: the application (app) layer, the application framework layer, the Android runtime and system libraries, and the kernel layer. The application layer may include a series of application packages.

[0137] like Figure 2 As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, etc.

[0138] The application framework layer provides an application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0139] like Figure 2 As shown, the application framework layer may include a window manager, an activity manager, a package manager, a resource manager, a view system, a phone manager, a notification manager, and the like.

[0140] Resource Manager, also known as Resource Management Service (RMS), provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0141] A window manager, also known as a window management service (WMS), manages windowed programs. It can determine the display size, determine whether a status bar is present, lock the screen, and take screenshots.

[0142] The activity manager, also known as the activity manager service (AMS), manages all application processes in the system.

[0143] The package manager, also known as the package management service (PMS), is responsible for functions such as application installation and uninstallation, component query and matching, and permission management.

[0144] The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build applications. The display interface can be composed of one or more views. For example, a display interface including a text message notification icon can include a view for displaying text and a view for displaying pictures.

[0145] The notification manager enables applications to display notification information in the status bar. It can be used to convey informative messages, which can automatically disappear after a short stay without user interaction. For example, the notification manager is used to inform that the download is complete, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of background running applications, and can also be a notification that appears on the screen in the form of a dialogue window. For example, it can prompt text information in the status bar, emit a prompt tone, vibrate the electronic device, and blink the indicator light, etc.

[0146] The system library can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing library (such as: OpenGL ES), 2D graphics engine (such as: SGL), etc.

[0147] The surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications.

[0148] The media library supports the playback and recording of multiple common audio and video formats, as well as static image files, etc. The media library can support multiple audio and video coding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0149] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing, etc.

[0150] The 2D graphics engine is a drawing engine for 2D drawing.

[0151] The kernel layer is the layer between hardware and software. The kernel layer contains at least a display driver, a camera driver, an audio driver, and a sensor driver.

[0152] Figure 3The figure shows a schematic diagram of a cross-device drag operation process provided by an embodiment of the present application, which involves a source device 10 and a target device 20, and the source device 10 and the target device 20 can be connected to each other. At a first moment, an operation object 30 can be displayed on the user interface of the source device 10. In response to the user continuously dragging the operation object from the user interface of the source device 10 to the target device 20, at a second moment, the operation object 30 can be displayed on the user interface of the target device 20.

[0153] Exemplarily, the source device 10 can be connected to a mouse, and the user can select and continuously drag the operation object 30 through the mouse to complete the operation of dragging the operation object 30 from the user interface of the source device 10 to the user interface of the target device 20. Similarly exemplarily, the source device 10 can include a touchpad, and the user can select and continuously drag the operation object 30 through the touchpad to complete the cross-device drag operation of the operation object 30 from the source device 10 to the target device 20.

[0154] In the scenario of the above cross-device drag operation, if the source device 10 is not connected to a mouse or does not include a touchpad, or, although the source device 10 includes a touchpad, the area of the operable area of the touchpad is too small to meet the requirement of the drag stroke for cross-device dragging, in these cases, it is difficult for the user to drag the operation object 30 from the source device 10 to the target device 20. The efficiency of the source device 10 and the target device 20 working together is relatively low, and the user experience of cross-device dragging is poor.

[0155] In order to improve the efficiency of collaborative task processing between multiple devices and enhance the user experience of cross-device drag operation, the present application provides a cross-device collaboration method, which is introduced as follows.

[0156] As Figure 4 The figure is another schematic diagram of a cross-device drag operation process provided by the present application. Still taking the source device 10 and the target device 20 as examples, the source device 10 and the target device 20 can be connected to each other.

[0157] An entry control 40A can be displayed on the user interface of the source device 10. The entry control 40A can be used to indicate the entry area for triggering the cross-device drag operation. In some scenarios, this entry area can also be referred to as a trigger area. Taking the operation object 30 on the user interface of the source device 10 as an example, in response to the user dragging the operation object 30 to the entry area, the operation object 30 can be displayed on the user interface of the target device 20.

[0158] In a scenario of cross-device dragging, the operation object 30 located at the entrance area of the starting device 10 can be directly displayed on the target device 20. The user can achieve cross-device dragging of the operation object 30 without performing a continuous dragging operation from the starting device 10 to the target device 20. The cross-device dragging operation can be independent of the hardware of the starting device 10. The implementation of this technical solution is beneficial to improving the efficiency of collaborative work between the starting device 10 and the target device 20 and enhancing the user experience of cross-device dragging operations.

[0159] In some examples, the starting device 10 and the target device 20 can be connected by wired or wireless means. Based on the established connection, the starting device 10 and the target device 20 can be used together, that is, the collaborative work described above. The starting device 10 and the target device 20 can be wirelessly connected using a variety of wireless communication protocols. Exemplarily, the wireless communication protocol can be a wireless fidelity (Wi-Fi) protocol, a Bluetooth protocol, a ZigBee protocol, a near field communication (NFC) protocol, various cellular network protocols, etc. This application does not limit this.

[0160] In some examples, the starting device 10 and / or the target device 20 can be a mobile phone, a tablet computer, a handheld computer, a PC, a cellular phone, a personal digital assistant (PDA), a wearable device (such as a smart watch), a smart home device (such as a television), a vehicle-mounted computer, a game console, and an augmented reality (AR) or virtual reality (VR) device, etc. The specific device form is not limited. For the introduction of the hardware architecture and software architecture of the starting device 10 and / or the target device 20, reference can be made to the relevant description of the electronic device 100 in the previous text. The device forms of the starting device 10 and the target device 20 can be the same or different. This application does not limit this. Exemplarily, the starting device 10 and the target device 20 can both be touch screen devices or both be non-touch screen devices.

[0161] The entry area of the starting device 10 can also be referred to as the trigger area or drag-sensing area for cross-device dragging. In response to a user's drag operation on an operation object on the user interface of the starting device 10, an entry control 40A can be displayed on the user interface of the starting device 10, and this entry control 40A can be used to indicate the aforementioned drag-sensing area. Similarly, the target device 20 can also be provided with a drag-sensing area. In response to a user's drag operation on an operation object on the user interface of the target device 20, a control similar to the aforementioned entry control 40A can also be displayed on the user interface of the target device 20, and this control can be used to indicate the drag-sensing area on the target device 20. The following takes the process of dragging an operation object on the starting device 10 to the target device 20 as an example to introduce the drag-sensing area.

[0162] In some examples, the drag-sensing area can be located on the outer periphery of the display area of the starting device 10, or rather, the aforementioned entry control 40A can be displayed on the outer periphery of the display area of the starting device 10. Exemplarily, as Figure 5 shown, the entry control 40A can include sub-controls 41A, 41B, 40C, and 40D. The drag-sensing area of the starting device 10 can include a first sub-area, a second sub-area, a third sub-area, and a fourth sub-area, and these four sub-areas are respectively located at the right edge, upper edge, left edge, and lower edge of the display area of the starting device 10. The four sub-controls of the entry control 40A can be respectively used to indicate the four sub-areas of the entry area, that is, the sub-control 41A can be used to indicate the first sub-area, the sub-control 41B can be used to indicate the second sub-area, the sub-control 41C can be used to indicate the third sub-area, and the sub-control 41D can be used to indicate the fourth sub-area.

[0163] In some examples, in response to a user's drag operation on the user interface of the starting device 10, the starting device 10 can simultaneously display the above four sub-controls.

[0164] In some examples, in response to a user's drag operation on the user interface of the starting device 10, the starting device 10 can display the sub-control corresponding to the operation direction in the drag-sensing area according to the operation direction (or called the drag direction) of the user's drag operation. Exemplarily, when the operation object is in the non-drag-sensing area of the display area of the starting device 10, in response to the user's operation of dragging the operation object to the right, the starting device 10 can display the sub-control 41A located at the right edge of the display area. In response to the user's operation of dragging the operation object downward, the starting device 10 can display the sub-control 41D located at the lower edge of the display area.

[0165] In a possible implementation, the starting device 10 can determine the operation direction of the user's dragging operation based on the capacitance information of its own display screen, etc. Exemplarily, the starting device 10 can obtain the change in capacitance values of different areas of the display screen during the user's dragging process, and determine the path of the user's dragging operation based on the change in capacitance values of different areas, so as to determine the operation direction of the user's dragging operation.

[0166] In some examples, in response to the user's dragging operation on the user interface of the starting device 10, the starting device 10 can display a sub-control corresponding to the position of the target device 20 in the drag perception area according to the relative spatial position relationship between the target device 20 and the starting device 10. Exemplarily, when the target device 20 is located on the right side of the starting device 10, in response to the user's dragging operation, the starting device 10 can display the sub-control 41A located at the right edge of the display area. When the target device 20 is located above the starting device 10, in response to the user's dragging operation, the starting device 10 can display the sub-control 41B located at the upper edge of the display area.

[0167] In a possible implementation, the starting device 10 can obtain the distance to other devices connected to the starting device 10 and the direction of other devices relative to the starting device 10 through positioning technologies such as Bluetooth, Ultra-wideband (UWB), and ultrasonic waves.

[0168] In some examples, the starting device 10 can display the sub-controls of the drag perception area by combining the operation direction of the user's dragging operation and the relative position relationship between the target device 20 and the starting device 10.

[0169] Exemplarily, the starting device 10 can establish connections with multiple target devices 20 at the same time, and the relative position relationships between the multiple target devices 20 and the starting device 10 are different. For example, the target device 20A is located on the right side of the starting device 10, the target device 20B is located above the starting device 10, and the target device 20C is located on the left side of the starting device 10. In this case, in response to the user's operation of dragging the operation object to the lower right of the operation object, the starting device 10 can display the sub-control 41A corresponding to the target device 20A located on the right side, and does not display the sub-control 41D.

[0170] In some examples, in response to the user's operation of starting to drag, the starting device 10 can display an entry control 40A for indicating the drag perception area. Here, the operation of starting to drag can refer to the operation of selecting the operation object and starting to continuously move the operation object.

[0171] In some examples, in response to the user dragging the operation object to the boundary position between the drag perception area and the non-drag perception area, the starting device 10 can display the aforementioned entry control 40A. Refer toFigure 5 The boundary between the drag-sensing area and the non-drag-sensing area can be composed of four boundary lines, namely boundary line 42A, boundary line 42B, boundary line 42C, and boundary line 42D. Exemplarily, in response to the user's operation of dragging the operation object onto boundary line 42A, the originating device 10 can display the sub-control 41A. Alternatively, in response to the user's operation of dragging the operation object onto boundary line 42A, the originating device 10 can simultaneously display the four sub-controls included in the drag-sensing area, namely sub-control 41A, sub-control 41B, sub-control 41C, and sub-control 41D.

[0172] The above technical solution can also be understood as that in response to the user's operation of dragging the operation object to a position where the distance from the boundary of the display area of the originating device 10 is less than or equal to the first distance threshold, the originating device 10 can display the entry control 40A for indicating the drag-sensing area. Exemplarily, in response to the user's operation of dragging the operation object to a position where the distance from the right boundary of the display area is less than or equal to the first distance threshold, the originating device 10 can display the sub-control 41A. Alternatively, in response to the user's operation of dragging the operation object to a position where the distance from the right boundary of the display area is less than or equal to the first distance threshold, the originating device 10 can simultaneously display the four sub-controls of the entry control 40A.

[0173] When the originating device 10 is connected to multiple target devices 20, the multiple target devices 20 can all be located on the same side of the originating device 10, or rather, the relative position relationship between the multiple target devices 20 and the originating device 10 is the same.

[0174] In some examples, in response to the user's operation of dragging the operation object, the originating device 10 can display the identification information of multiple target devices 20 located in the same direction as the originating device 10 in the sub-area corresponding to the target device 20. In response to the user's operation of dragging the operation object 30 to the display area of the identification information of one of the target devices 20, the target device 20 can display the operation object 30.

[0175] Exemplarily, as Figure 6 shown, the target devices 20A, 20B, and 20C are all located above the originating device 10. In response to the user's operation of dragging the operation object 30, the originating device 10 can display the identification information of the target devices 20A, 20B, and 20C at the position of the sub-area 41B. In response to the user's operation of dragging the operation object 30 to the display area of the identification information of the target device 20B, the originating device 10 sends the information of the operation object 30 to the target device 20, so that the target device 20B can display the operation object 30.

[0176] In some examples, in response to a user's operation of dragging the operation object 30, the starting device 10 may display a recommended device in the sub-region corresponding to the target device 20. The recommended device may be determined based on one or more of the distances between multiple target devices 20 in the same direction as the starting device 10 and the starting device 10, the user's usage habits of using multiple target devices 20, the type of the operation object 30, and the types of multiple target devices 20, etc.

[0177] Exemplarily, taking Figure 6 the distance relationship between the target device 20A, the target device 20B, and the target device 20C and the starting device 10 in [Example] as an example, the distance between the target device 20A and the starting device 10 is the smallest, and the above-mentioned recommended device may be the target device 20A.

[0178] Exemplarily, among the target device 20A, the target device 20B, and the target device 20C, the user uses the starting device 10 and the target device 20C more frequently at the same time. In this case, the above-mentioned recommended device may be the target device 20C.

[0179] Exemplarily, when the operation object 30 is a video file, among the target device 20A, the target device 20B, and the target device 20C, the display effect of the target device 20A is the best. In this case, the above-mentioned recommended device may be the target device 20A.

[0180] In some examples, in response to a user's operation of dragging the operation object 30, the starting device 10 may display the identification information of the recommended device in the sub-region corresponding to the target device 20. The recommended device may be determined based on the relative positions of multiple target devices 20 and the starting device 10 and the dragging direction of the user's dragging operation.

[0181] In some examples, the relative positions of the identification information of multiple recommended devices are the same as the relative positions of multiple recommended devices.

[0182] In some examples, the connection direction between the target device 20 and the starting device 10 is parallel or approximately parallel to the dragging direction.

[0183] Exemplarily, taking Figure 7 the relative positions of the target device 20A, the target device 20B, and the target device 20C and the starting device 10 in [Example] as an example, the target device 20A is located in the upper left of the starting device 10, the target device 20B is located directly above the starting device 10, and the target device 20C is located in the upper right of the starting device 10.

[0184] In response to the user's operation of dragging the operation object 30 to the upper right of the user interface of the starting device 10, the starting device 10 can determine that the target device 20C located in the upper right of the starting device 10 is the recommended device, and can display the identification information of the target device 20C at the position of the sub-region 41B.

[0185] In response to the user's operation of dragging the operation object 30 to the upper left of the user interface of the starting device 10, the starting device 10 can determine that the target device 20A located in the upper left of the starting device 10 is the recommended device, and can display the identification information of the target device 20A at the position of the sub-region 41B.

[0186] In response to the user's operation of dragging the operation object 30 directly above the user interface of the starting device 10, the starting device 10 can determine that the target device 20B located directly above the starting device 10 is the recommended device, and can display the identification information of the target device 20B at the position of the sub-region 41B.

[0187] In some examples, in response to the user's operation of dragging the operation object 30, the starting device 10 can display the recommended device at the sub-region corresponding to the target device 20, and the recommended device can be determined according to the relative positions of multiple target devices 20 and the starting device 10 and the postures of multiple target devices 20.

[0188] Here, the posture of the target device 20 can also be understood as the placement posture of the target device 20 or the usage posture of the target device 20, for example: placed flat, placed vertically, placed obliquely, placed upside down (with the screen facing the desktop), etc.

[0189] Exemplarily, taking Figure 7 the relative positions and placement postures of the target device 20A, the target device 20B, and the target device 20C in as an example, the target device 20A, the target device 20B, and the target device 20C are all located above the starting device 10, the target device 20A is placed flat, the target device 20B is placed vertically, and the target device 20C is placed obliquely with the screen tilted towards the starting device.

[0190] In a possible implementation manner, the starting device can display the placement postures of multiple target devices 20 in the drag sensing area, or rather, the identification of the device displayed in the drag sensing area can also be used to indicate the placement posture of the target device 20.

[0191] Among the multiple target devices 20 above, the posture of the target device 20C is more convenient for the user to view the result after cross-device dragging. In response to the user's operation of dragging the operation object 30 above the user interface of the starting device 10, the starting device 10 can determine the target device 20C in an inclined posture as the recommended device, and can display the identification information of the target device 20C at the position of the sub-region 41B.

[0192] In some examples, in response to the user's operation of dragging the operation object 30, the starting device 10 can display the recommended device at the sub-region corresponding to the target device 20. The recommended device can be determined according to one or more of the relative position relationship between the multiple target devices 20 and the starting device 10, the dragging direction of the user's dragging operation, and the placement posture of the device. Exemplarily, the recommended device can be jointly determined by combining the three factors of the relative position relationship between the multiple target devices 20 and the starting device 10, the dragging direction of the user's dragging operation, and the placement posture of the device.

[0193] In the case where there is only one recommended device determined by the starting device 10 above, in response to the user's operation of dragging the operation object 30, the starting device can also not display the identification information of the recommended device at the sub-region. The recommended device can directly display the dragged operation object 30 or the user interface including the operation object 30.

[0194] In a possible implementation manner, the starting device 10 can determine the placement postures of the multiple target devices 22 according to signals such as magnetic fields emitted by the multiple target devices 22. Exemplarily, the application number 202310666025.4 provides a method for determining the placement posture of an electronic device. It should be noted that this application does not limit the method for determining the placement posture of the target device 22.

[0195] For the case where the identification information of the multiple target devices 20 needs to be displayed simultaneously, the display manner and display position of the identification information of the multiple target devices 20 can be determined according to one or more of the distances between the multiple target devices 20 in the same direction as the starting device 10 and the starting device 10, the user's usage habits of the multiple target devices 20, the type of the operation object 30, and the types of the multiple target devices 20.

[0196] Figure 9 Exemplarily shows a display manner of the identification information of the multiple target devices 20. Different from Figure 6 the display manner of the identification information of the multiple target devices 20 in Figure 6 where the area sizes of the display regions corresponding to the identification information of different target devices 20 are all equal, Figure 9 in

[0197] In some examples, in response to a user's drag operation, the originating device 10 may display the identification information of the target device 20 with a higher priority at the position of the display area 45A, display the identification information of the target device 20 with a lower priority at the position of the display area 45B, and display the identification information of the target device 20 with the lowest priority at the display area 45C.

[0198] Exemplarily, taking Figure 6 the distance relationship between the target device 20A, the target device 20B, and the target device 20C and the originating device 10 as an example, the distance between the target device 20A and the originating device 10 is the smallest, and the distance between the target device 20C and the originating device 10 is the largest. In this case, the originating device 10 may display the identification information of the target device 20A at the position of the display area 45A, display the identification information of the target device 20B at the position of the display area 45B, and display the identification information of the target device 20C at the position of the display area 45C.

[0199] Exemplarily, among the target device 20A, the target device 20B, and the target device 20C, the user uses the originating device 10 and the target device 20C the most times simultaneously, and the user uses the originating device 10 and the target device 20B the fewest times simultaneously. In this case, the originating device 10 may display the identification information of the target device 20C at the position of the display area 45A, display the identification information of the target device 20A at the position of the display area 45B, and display the identification information of the target device 20B at the position of the display area 45C.

[0200] Exemplarily, when the operation object 30 is a painting application, both the target device 20A and the target device 20B support touch operations, the target device 20C does not support touch operations, and the target device 20A also supports stylus operations. The originating device 10 may determine the display positions of the identification information of different target devices 20 according to the application scenarios of the current user's device usage or the application scenarios of multi-device collaboration. For example, the originating device 10 may display the identification information of the target device 20A that supports stylus operations at the position of the display area 45A, display the identification information of the target device 20B that supports touch operations at the position of the display area 45B, and display that the target device 20C does not support touch operations at the position of the display area 45C.

[0201] In some examples, the multiple regions that display the identification information of the multiple target devices 20 in the above examples may form a sub-region of the aforementioned drag perception region (such as the aforementioned first sub-region or second sub-region, etc.). Or rather, the sub-region of the drag perception region may include multiple regions for displaying the identification information of the multiple target devices 20. Or rather, the interface component for displaying the identification information of the multiple target devices 20 and the interface component for displaying the sub-region of the drag perception region are the same interface component (such as sub-control 41A or sub-control 41B, etc.). Or rather, the identification information of the multiple target devices 20 may be displayed on the sub-control of the aforementioned entry control 40A. Exemplarily, Figure 6 the identification information of 3 target devices may be displayed on sub-control 41B.

[0202] In some examples, the interface component for displaying the identification information of the multiple target devices 20 is a device display control different from the entry control 40A. The starting device 10 may display the above device display control after displaying the entry control 40A. Exemplarily, in response to the user's operation of dragging the operation object 30, the starting device 10 may be used to indicate sub-control 41B of the second sub-region. In response to the user's operation object 30 remaining in the second sub-region for a duration greater than the first time threshold, the starting device 10 may display Figure 6 the device display control, which may be used to display the identification information of target device 20A, the identification information of target device 20B, and the identification information of target device 20C.

[0203] In some examples, the aforementioned operation object 30 may be one or more interface elements or the content of interface elements displayed on the user interface of the starting device 10. Exemplarily, the operation object 30 may include one or more of the following: text, picture, code, file, interface component, and application window, etc. Among them, the interface component may include buttons, text boxes, sliders, and icons, etc. The application window may include full-screen windows, split-screen windows, or floating windows, etc.

[0204] For different types of operation objects 30, different operation bodies may trigger drag events. Taking the starting device 10 as a touch screen device as an example, in some examples, fingers and styli may be used to trigger drag events for different operation objects 30. Here, fingers and styli may be regarded as two different operation bodies.

[0205] Taking the picture editing scenario as an example, long pressing with a finger or a stylus may trigger the drag event of the picture to be edited. Since a pressure sensor may be equipped in the stylus, the heavy pressing operation of the stylus may be used to trigger the drag events of picture elements such as colors and shapes in the picture to be edited.

[0206] In some examples, finger operations may include: single-finger operations, multi-finger operations, palm operations, or knuckle operations, etc. Among them, operations of different fingers in single-finger operations can be used to trigger drag events of different operation objects 30. Multi-finger operations may include two-finger operations, three-finger operations, or four-finger operations, etc. Multi-finger operations with different numbers can also be used to trigger drag events of different operation objects 30.

[0207] Taking the text editing scenario as an example, single-finger operations can be used to trigger drag events of text in the file to be edited. Three-finger operations can be used to trigger drag events of screenshots of the text editing interface. Palm operations can be used to trigger drag events of the text editing window.

[0208] In some examples, for the same operation body, different drag events of different operation objects 30 can also be triggered according to different operation methods. For example, single-finger long press can be used to trigger drag events of recognizable objects in an image, and single-finger long hold can be used to trigger drag events of the image.

[0209] Taking Figure 10 the picture editing scenario in as an example, in the left schematic diagram, in response to the user's single-finger tap operation on the picture to be edited, the starting device 10 can determine that the operation object 30 is the picture to be edited. For the sake of distinction, it is hereinafter referred to as operation object 31A. In the right schematic diagram, in response to the user's single-finger long press operation on the picture to be edited, the starting device 10 can determine that the operation object is the recognizable object in the picture to be edited. For the sake of distinction, it is hereinafter referred to as operation object 31B.

[0210] In response to the drag operation on different types of operation objects 30, the starting device 10 can display the processing method corresponding to this type of operation object 30 or the application method of the target device 20 corresponding to this type of operation object 30 at the position of the drag sensing area. For example, the application methods of the target device 20 may include mirroring, stretching, and sharing, etc.

[0211] In some scenarios, the application method of the device can also be understood as the display method of the operation object.

[0212] Exemplarily, as Figure 11 shown, in response to the user's operation of dragging the application window in the user interface of the starting device 10, the starting device 10 can display three application methods of the target device 20 at the position of the sub-region 41B, namely method 1A, method 1B, and method 1C.

[0213] Among them, Mode 1A may refer to using the target device 20 as a mirror display device, that is, the content displayed on the starting device 10 and the target device 20 is the same. In this case, in response to the user's drag operation of dragging the application window to the display area of Mode 1A, the starting device 10 and the target device 20 can simultaneously display the application window dragged by the user, and the states of the application windows displayed on the two devices are the same.

[0214] Mode 1B may refer to using the target device 20 as an extended display device, that is, the display screens of the starting device 10 and the target device 20 jointly form the display area of the starting device 10. In this case, in response to the user's drag operation of dragging the application window to the display area of Mode 1B, the target device 20 can display the application window dragged by the user, and the starting device 10 may not display the application window dragged by the user.

[0215] Mode 1C may refer to using the target device 20 as a data sharing device, and the data of the starting device 10 can be transmitted to the target device 20. In this case, in response to the user's drag operation of dragging the application window to the display area of Mode 1C, the starting device 10 can send information related to the application window to the target device 20, and the target device 20 can display files and the like for indicating the information of the foregoing application window.

[0216] In a possible implementation manner, before the user performs the drag operation, the application mode of the target device 20 has been set, or rather, the application mode of the target device 20 is set with a default value. In response to the user's drag operation, the starting device 10 can perform the drag operation on the operation object 30 according to the set application mode of the target device 20. Exemplarily, the application mode of the target device 20 can be set to the foregoing Mode 1B or the "extended" application mode. In response to the user's operation of dragging the operation object 30, the target device 20 can display the operation object 30, and the starting device 10 may not display the operation object 30.

[0217] In a possible implementation manner, before the user performs the drag operation, the application mode of the target device 20 has been set. In response to the user's drag operation and the operation of reselecting the application mode of the target device 20, the starting device 10 can perform the drag operation on the operation object 30 according to the application mode of the target device 20 reselected by the user. Exemplarily, in response to the user's operation of dragging the operation object 30 to the drag sensing area and staying for more than the second time threshold, the starting device 10 can display one or more application modes of the target device 20 at the position of the drag sensing area. In response to the user's operation of dragging the operation object to the display area of the foregoing one or more application modes, the starting device 10 can determine the application mode of the target device 20 reselected by the user and perform the drag operation on the operation object 30 according to the reselected application mode of the user.

[0218] In some examples, in response to the user dragging an operation object to the display area of the foregoing method 1A, method 1B, or method 1C, the starting device 10 may send indication information to the target device 20, and the indication information is used to indicate the display mode of the operation object selected by the user or the application mode of the target device 20.

[0219] During each cross-device drag operation, the user can re-select the application mode of the target device 20, which is beneficial to improving the efficiency of collaborative work between the starting device 10 and the target device 20 to a certain extent and enhancing the user experience of the cross-device drag operation.

[0220] As described above, when the operation object 30 is moved to the drag sensing area of the starting device 10, the target device 20 may display the operation object 30.

[0221] In some examples, before the operation object 30 is moved to the drag sensing area of the starting device 10, in response to the user stopping dragging or releasing the operation object 30, the operation object 30 may return to the initial position. Exemplarily, referring to Figure 4 , in response to the user releasing the operation object 30 before moving the operation object 30 to the entrance area, the starting device 10 may display the operation object 30 at the initial position of the operation object 30 indicated by the dashed line, while the target device 20 does not display the operation object 30, or rather, the target device 20 does not perform operations related to the operation object 30.

[0222] Continuing to refer to Figure 4 , the user interface of the target device 20 may include an exit area, or rather, the target device 20 may display an exit control 40B, and the exit control 40B may be used to indicate the exit area on the target device 20. In some scenarios, the exit area may also be referred to as a response area. Similar to the drag sensing area of the foregoing starting device 10, the exit area may be located at the edge of the display area of the target device 20. In other words, the exit control 40B may be located on the outer periphery of the display area of the target device 20.

[0223] In some examples, the exit area of the target device 20 may include a plurality of sub-areas (such as 4) similar to the drag sensing area of the Figure 5 starting device 10, and the plurality of sub-areas may be displayed at the edges of the four sides of the display area of the target device 20. Or rather, the exit control 40B may include a plurality of sub-controls, and the plurality of sub-controls may be respectively used to indicate the plurality of sub-areas of the exit area, and the plurality of sub-controls may be respectively displayed around the display area of the target device 20.

[0224] In some examples, the exit control 40B of the target device 20 may be displayed only on the edge of the display area of the target device 20 near the starting device 10. For Figure 4 taking the relative position relationship between the starting device 10 and the target device 20 as an example, if the target device 20 is located on the right side of the starting device 10, the exit control 40B of the target device 20 may be displayed on the left edge of the display area.

[0225] In some examples, the target device 20 may display the operation object 30 at the position of the aforementioned exit area.

[0226] Exemplarily, the target device 20 may prominently display the operation object 30 at the position of the exit area to remind the user. For example, the target device 20 may display the operation object 30 in a highlighted color, the target device 20 may also display the operation object 30 with animation effects (such as jittering, deformation, etc.), the target device 20 may also magnify the display of the operation object 30, and the target device 20 may also hover and display the operation object 30 at the position of the exit area.

[0227] For the case where the target device 20 displays the operation object 30 at the position of the exit area, in some examples, as Figure 12 shown, in response to the user's operation of continuously dragging the operation object 30 on the target device 20, the target device 20 may display the operation object at the target position 30D selected by the user. Figure 12 In this example, the target object 30 is dragged from the initial position 30A on the starting device 10 to the entrance position 30B of the exit area, thereby triggering cross-device dragging, and the operation object 30 is displayed at the exit position 30C of the exit area of the target device 20; in response to the user's continuous dragging operation on the target device 20, the target device 20 may display the operation object 30 at the target position 30D.

[0228] In some examples, the entrance position 30B may be the position where the user stops dragging the operation object 30 and releases it.

[0229] In some examples, in response to the user's operation of continuously dragging the operation object 30 from the initial position 30A to the drag sensing area until the edge of the display area of the starting device 10, the starting device 10 may determine the position information of the dragged object 30 leaving the display area. For example, the coordinate information of the position when the dragged object 30 leaves the display area. In this case, the position where the dragged object 30 leaves the display area may be regarded as the aforementioned entrance position 30B.

[0230] The position where the operation object 30 is displayed on the target device 20, the exit position 30C, can be determined based on information such as the entry position 30B. Exemplarily, the distances between the exit position 30C and the upper and lower edges of the target device 20 are L1 and L2 respectively, and the distances between the entry position and the upper and lower edges of the starting device 10 are L3 and L4 respectively. Then, L1:L2 = L3:L4.

[0231] For example, the width of the display area of the starting device 10 is set as w1, and the width of the display area of the target device 20 is set as w2. Taking the length direction of the device as the x-axis and the width direction as the y-axis, with the lower left corner of the display area as the coordinate origin, the coordinates of the entry position 30B of the operation object 30 on the starting device 10 are set as (x1, y1), and the coordinates of the exit position 30C of the operation object 30 on the target device 20 are set as (x2, y2). Taking the example that the target device 20 is located on the right side of the starting device 10 and the user drags the operation object 30 from the initial position 30A to the lower right of the starting device 10, then y2 = y1*w2 / w1 and x2 = 0.

[0232] In some examples, the starting device 10 can send indication information to the target device 20, and this indication information can be used to indicate the entry position, which can be the position where the operation object 30 is released in the trigger area.

[0233] In some examples, in response to the user's operation of dragging the operation object 30 to the target application on the target device 20, the target device 20 can display the user interface of the target application, and this target application can be used to open or process the operation object 30.

[0234] Here, the target application to which the operation object 30 is dragged can be understood as the user interface of the target application, the icon of the target application, or other interface elements that can trigger the invocation of the target application, etc.

[0235] Exemplarily, as Figure 13 shown, the operation object 30 can be a picture file. In response to the user's operation of dragging the picture file to the application icon of the "Gallery" application 50, the target device 20 can display Figure 13 the user interface 51 of the "Gallery" application 50 shown below, and this user interface 51 can be used to display the aforementioned picture file.

[0236] Exemplarily, the operation object 30 can be text information. In response to the user's operation of dragging the text information to the user interface of the "Text Editor" application, the target device 20 can display another user interface of the "Text Editor" application, and the aforementioned text information dragged by the user can be displayed on this user interface.

[0237] The operation object 30 located at the initial position 30A above can also be referred to as the operation object 30 in the starting state. The operation object 30 located at the entrance position 30B can also be referred to as the operation object 30 in the entrance state. The operation object 30 located at the exit position 30B can also be referred to as the operation object 30 in the exit state. The operation object 30 located at the target position 30D can also be referred to as the operation object 30 in the target state.

[0238] For the case where the target device 20 displays the operation object 30 in the exit area, in some examples, such as Figure 14 As shown, in response to the operation object 30 not being operated within a preset duration, the target device 20 may no longer display the operation object 30. In other words, the target device 20 can return to the state before performing the cross-device drag operation. The starting device 10 can display the operation object 30 at the initial position 30A, or rather, the starting device 10 can display the operation object 30 in the initial state. In other words, the starting device 10 can return to the state before performing the cross-device drag operation.

[0239] In some examples, the operation object 30 can be displayed on the default interface of the target device 20. Or rather, the operation object 30 can be saved at the position of the default address. The default interface or default address here can be set by the user, or can also be preset by the target device 20. And for different types of operation objects 30, the default interface or default address can be set separately.

[0240] Exemplarily, such as Figure 15 As shown, the operation object 30 can be an application window, and the default interface can be the interface currently displayed on the target device 20. In other words, in response to the user's operation of dragging the application window from the starting device 10 to the target device 20 across devices, the target device 20 displays the user interface of the application window on top of the current user interface.

[0241] For the case where the operation object 30 is an application window, according to the type of the target device 20, the target device 20 can display the application window in different preset ways.

[0242] Exemplarily, the target device 20 can determine different ways to display the user interface corresponding to the application window according to the size of its own display screen. When the screen size of the target device 20 is greater than the first area threshold, the target device 20 can display the user interface of the operation object 30 in the form of a floating window. When the screen size of the target device 20 is less than or equal to the first area threshold, the target device 20 can display the user interface of the operation object 30 in the form of a full-screen window. Exemplarily, the aforementioned first area threshold can be represented by the diagonal length of the screen. For example, the first area threshold is 8 inches.

[0243] For example,Figure 15 In this case, the length of the diagonal of the screen of the target device 20 is 11 inches, and in this situation, the target device 20 can display the user interface of the operation object 30 in the form of a floating window.

[0244] For example, Figure 16 In this case, the length of the diagonal of the screen of the target device 20 is 6.7 inches, and in this situation, the target device 20 can display the user interface of the operation object 30 in the form of a full-screen window.

[0245] For the case where the operation object 30 is an application window, according to the type of the target device 20, the target device 20 can determine whether to display different windows of the application corresponding to the operation object 30 in a split-screen display manner.

[0246] Exemplarily, the application window corresponding to the operation object 30 may include a first window and a second window. When the screen size of the display screen of the target device 20 is greater than or equal to the second threshold, in response to the user's operation of dragging the operation object across devices, the target device 20 can display the first window and the second window. When the screen size of the display screen of the target device 20 is less than the second threshold, in response to the user's operation of dragging the operation object across devices, the target device 20 can display only the first window or the second window. In some examples, when the operation object 30 is a desktop widget, a quick application, or a desktop component, etc., in response to the user's operation of performing cross-device dragging, the target device 20 can also display a window corresponding to the operation object 30 in the current user interface.

[0247] Exemplarily, the operation object 30 may be a target file, and the default interface may be the interface of an application program that can open the target file. In other words, in response to the user's operation of dragging the target file from the starting device 10 across devices to the target device 20, the target device 20 can display the user interface of the target application, and the target application can be used to open or process the target file.

[0248] For example, the operation object 30 may be a picture file, and the default interface may be the user interface 51 of the "gallery" application 50 mentioned above Figure 13 in which the user interface 51 can be used to display the aforementioned picture file. In other words, in response to the user's operation of dragging the picture file from the starting device 10 across devices to the target device 20, the target device 20 can display the user interface for displaying the picture file.

[0249] For another example, the operation object 30 may be text information, and the default interface may be the user interface of a "text editing" application, which may display the aforementioned text information dragged by the user. In other words, in response to the user's operation of dragging the text information from the source device 10 across devices to the target device 20, the target device 20 may display the user interface of the application for editing the text information.

[0250] In some examples, at the position of the exit area 40B, the target device 20 may display one or more processing methods corresponding to the operation object 30 or the application methods of the target device 20 corresponding to this type of operation object 30. For example, the application methods of the target device 20 may include mirroring, extension, and sharing, etc.

[0251] Exemplarily, as Figure 17 , in response to the user's operation of dragging the application window in the user interface of the source device 10 to the drag perception area, the target device 20 may display the operation object 30. For example, the target device 20 may display the operation object 30 at the center position of the user interface. In response to the user's operation of selecting the operation object 30, the target device 20 may display three application methods of the target device 20 at the position of the exit area 40B, namely, method 1A, method 1B, and method 1C.

[0252] Among them, method 1A may refer to using the target device 20 as a mirror display device, that is, the content displayed on the source device 10 and the target device 20 is the same. In this case, in response to the user's dragging operation of dragging the application window to the display area of method 1A, the source device 10 and the target device 20 may simultaneously display the application window dragged by the user, and the states of the application windows displayed on the two devices are the same. Method 1B may refer to using the target device 20 as an extended display device, that is, the display screens of the source device 10 and the target device 20 jointly form the display area of the source device 10. In this case, in response to the user's dragging operation of dragging the application window to the display area of method 1B, the target device 20 may display the application window dragged by the user, and the source device 10 may not display the application window dragged by the user. Method 1C may refer to using the target device 20 as a data sharing device, and the data of the source device 10 may be transmitted to the target device 20. In this case, in response to the user's dragging operation of dragging the application window to the display area of method 1C, the source device 10 may send information related to the application window to the target device 20, and the target device 20 may display files and the like for indicating the information of the aforementioned application window.

[0253] For different types of operation objects 30, or rather, for different types of cross-device dragging tasks, or rather, for different application methods of the target device 20, the target device 20 may display the user interface of the operation object 30 in different ways.

[0254] In some examples, the operation object 30 can be the window of an application that needs to be displayed in real time. In response to the user's cross-device dragging operation, the target device 20 can display the window of the application. An example was given above Figure 16 That is an example. In some examples, the operation object 30 can be a data file such as a picture file or a video file. During the transmission process of these data files, the target device 20 may not display them in real time. In this case, in response to the user's cross-device dragging operation, the target device 20 can continue to display the currently displayed user interface. Or rather, the target device 20 may not display the transmission process of the operation object 30 in real time.

[0255] In some examples, a screen sharing task is performed between the source device 10 and the target device 20. The user's cross-device dragging operation on the operation object 30 can trigger this screen sharing task. In this case, in response to the user's cross-device dragging operation, the target device 20 can display the content shown on the screen of the source device 10. Or rather, the target device 20 can display the transmission process of the cross-device dragged object in real time. In some examples, a data sharing task is performed between the source device 10 and the target device 20. The user's cross-device dragging operation on the operation object 30 can trigger this data sharing task. In this case, in response to the user's cross-device dragging operation, the target device 20 may not display the transmission process of the operation object 30 in real time.

[0256] In some examples, in response to the user selecting "Mode 1A" or "Mode 1B" described above as the application mode of the target device 20 during the cross-device dragging operation. Or rather, in response to the user choosing to apply the target device 20 in a mirror or extended manner, the target device 20 can display the transmission process of the cross-device dragged operation object 30 in real time. In some examples, in response to the user selecting "Mode 1C" described above as the application mode of the target device 20 during the cross-device dragging operation. Or rather, in response to the user choosing to apply the target device 20 in a shared manner, the target device 20 may not display the transmission process of the cross-device dragged operation object 30 in real time.

[0257] In some scenarios, the above technical solution can also be understood in the following way: For the case where the operation object 30 involves (or includes) a synchronous transmission task, the target device 20 can explicitly display the result of the transmission of the operation object 30. For the case where the operation object 30 does not involve (or does not include) a synchronous transmission task, the target device 20 may not explicitly display the transmission result of the operation object 30. Or rather, for the case where the operation object 30 only involves (or only includes) an asynchronous transmission task, the target device 20 may not explicitly display the transmission result of the operation object 30.

[0258] Taking the application mode "Mode 1A" of the target device 20 as an example, the content displayed on the display screen of the starting device 10 can be displayed on the display screen of the target device 20. In other words, the starting device 10 can project and display the content on the target device 20. In this case, the target device 20 can be regarded as the transmission result of the explicit display operation object 30.

[0259] Reference Figure 18 , where the operation object 30 is text information. In response to the user's operation of dragging the operation object 30 to the "Memo" application desktop plugin on the user interface of the target device 20, the target device 20 can create a new memo content including the text information in the desktop plugin. In this case, the target device 20 can be regarded as the transmission result of the non-explicit display operation object 30.

[0260] Displaying the operation object of the cross-device drag operation in different ways is beneficial to more reasonably scheduling the software and hardware resources of the target device 20 and improving the cooperation efficiency between the starting device 10 and the target device 20.

[0261] In the above embodiment, in response to the user's operation of dragging the operation object 30 on the starting device 10 to the entrance area and stopping dragging or releasing the operation object 30, the starting device 10 can trigger the task of performing the cross-device drag operation, and then cooperate with the target device 20 to achieve coordination. The implementation of this technical solution can, to a certain extent, reduce the adverse impact of the continuous drag distance on the cross-device drag operation and is beneficial to improving the cooperation efficiency between the starting device 10 and the target device 20.

[0262] In some scenarios, the starting device 10 and the target device 20 can be placed side by side. In this case, the success rate of the cross-device drag operation can be improved by determining the user's intention to continuously drag on the starting device 10. Or rather, in this case, the success rate of the cross-device drag operation can be improved by determining whether the user needs to perform the cross-device drag operation.

[0263] Figure 19 Shown is a method for cross-device cooperation provided by an embodiment of the present application. In the case where an operation object is dragged to the trigger area of a first device, a second device connected to the first device can display the operation object. In this method, the user does not need to continuously drag the operation object from the first device to the second device to trigger the cross-device drag of the operation object, and this method is beneficial to improving the cooperation efficiency between the first device and the second device.

[0264] S101, in response to the drag operation on the operation object on the first device, move the operation object.

[0265] The operation object can be one or more interface elements or the content of interface elements on the start interface of the user interface of the first device. Exemplarily, the operation object can include one or more of the following: text, picture, code, file, interface component, application window, etc. Among them, the interface component can include buttons, text boxes, sliders, icons, etc. The application window can include full-screen window, split-screen window, floating window, etc.

[0266] The object used for dragging the operation object can be called the operation body. The operation body can be a mouse, touchpad, digitizing tablet, finger, stylus, etc. Different numbers of the same operation body can also be regarded as different operation bodies. In some examples, different fingers of the user, different numbers of fingers can all be regarded as different operation bodies. Different operation bodies can be used to trigger the dragging operations of different operation objects.

[0267] Exemplarily, in a picture editing interface, a finger can trigger the dragging operation of the picture to be edited, and a stylus can trigger the dragging operation of picture elements such as colors and lines in the picture to be edited.

[0268] Different operation methods can be used to trigger the dragging operations of different types of operation objects. Here, the operation methods can include: tapping, long pressing, hard pressing, double tapping, etc.

[0269] Exemplarily, in a picture editing interface, finger tapping can trigger the dragging operation of the picture editing window, finger long pressing can trigger the dragging operation of the picture to be edited, and finger hard pressing can trigger the dragging operation of recognizable objects in the picture to be edited.

[0270] In some examples, the direction in which the user moves the operation object can become the operation direction. In one possible implementation, the first device can determine the operation direction of the user's dragging operation according to the capacitance information of its own display screen, etc. The first device can determine the second device according to the operation direction.

[0271] S102, determine whether the operation object is in the trigger area on the first device.

[0272] The trigger area can refer to the area on the first device that can trigger the cross-device dragging of the operation object. The trigger area can be the dragging perception area in the foregoing embodiments.

[0273] In some examples, the trigger area can be located on the outer periphery of the display area of the first device. Exemplarily, the foregoing dragging perception area can include a first sub-area, a second sub-area, a third sub-area, and a fourth sub-area.

[0274] The first device can indicate the foregoing trigger area through an indication control. The indication control can include a plurality of sub-controls, and the plurality of sub-controls can be respectively used to indicate different positions of the trigger area.

[0275] Exemplarily, the indication control may be the aforementioned entry control 40A, which can be used to indicate the entry area. The entry control 40A may include sub-controls 41A, 41B, 41C, and 41D. The sub-control 41A can be used to indicate the first sub-area, the sub-control 41B can be used to indicate the second sub-area, the sub-control 41C can be used to indicate the third sub-area, and the sub-control 41D can be used to indicate the fourth sub-area.

[0276] In some examples, in response to the user's operation of dragging an operation object, the first device may display an indication control around the display area, and the indication control can be used to indicate that the trigger area is around the display area of the first device.

[0277] In some examples, in response to the user's operation of dragging an operation object, the first device may display an indication control at the edge of the side of the display area close to the second device, and the indication control can be used to indicate that the trigger area is at the edge of the side of the display area of the first device close to the second device.

[0278] In some examples, in response to the user's operation of dragging the operation object in the first direction, the first device may display an indication control at the edge of the display area in the direction indicated by the first direction. Exemplarily, if the first direction is to the right, the first device may display an indication control at the position of the right edge of the display area. Exemplarily, if the first direction is to the upper left, the first device may display indication controls at the positions of the upper edge and the left edge of the display area.

[0279] The distance between the boundary of the trigger area and the area outside the trigger area and the outer boundary of the display area of the first device can be referred to as the first distance threshold. Or rather, the width of the trigger area can be the first distance threshold. In some examples, in response to the user's operation of dragging the operation object to a distance less than or equal to the first distance threshold from the outer boundary of the display area of the first device, the first device may display the aforementioned indication control to indicate the trigger area.

[0280] In some examples, indication information for indicating one or more application modes of the second device may also be displayed at the position of the trigger area. For example, the application modes of the second device may include mirroring, extension, sharing, etc.

[0281] Exemplarily, in response to the user's operation of dragging the operation object, the first device may display one or more application modes of the second device at the position of the trigger area. In response to the user's operation of dragging the operation object to the display area of one of the application modes, the second device displays the operation object according to the application mode selected by the user.

[0282] In some examples, the second device may be based on the relative position relationship between the second device and the first device, the placement posture of the second device, or the direction in which the operation object is dragged.

[0283] In some examples, the first device can establish connections with multiple electronic devices, and the location of the trigger area can also display the identification information of multiple second devices connected to the first device. The multiple second devices can be electronic devices with the same location relationship as the first device. The second device can be determined based on one or more of the distance between the multiple second devices and the first device, the user's usage habits of the multiple second devices, the type of the operation object, and the type of the multiple second devices. For details, please refer to the relevant description above and will not be repeated here.

[0284] In one possible implementation, the above-mentioned indication control for indicating the trigger area, the display control for displaying indication information indicating the application mode of the second device, and the display control for displaying identification information of multiple second devices can be the same control, or these controls can be multiple independent controls, which is not limited in this application.

[0285] S103: Display the operation object on the second device.

[0286] The second device may refer to an electronic device that establishes a connection with the first device, and the connection method may be a wired connection or a wireless connection, wherein the wireless connection protocol may include one or more of the following: Wireless Fidelity protocol, Bluetooth protocol, ZigBee protocol, short-range wireless communication protocol or various cellular network protocols, etc.

[0287] In some examples, in response to a user dragging an operation object to a trigger area of the first device, the operation object is displayed on the user interface of the second device.

[0288] The second device may be the recommended device determined in the aforementioned embodiment based on the distance from the first device, the user's usage habits of multiple second devices, the type of the operation object, and the types of the multiple second devices. Alternatively, the second device may be selected by the user based on identification information of multiple second devices displayed on the first device.

[0289] In other words, in response to an operation object being dragged to the display area of the identification information of the second device on the user interface of the first device, the operation object is displayed on the user interface of the second device. Alternatively, in response to an operation object being dragged to a trigger area on the first device near the side of the second device, the operation object is displayed on the user interface of the second device.

[0290] In some examples, the second device may display the operation object in the response area. Specifically, the second device may display the operation object at the exit position of the response area, and the exit position may be determined according to the entry position of the operation object on the first device. The entry position may be the position where the operation object is located when cross-device dragging is triggered on the first device, or the entry position may also be the position where the operation object leaves the display area of the first device. The method for determining the exit position according to the entry position may refer to the relevant descriptions in the previous text and will not be elaborated here.

[0291] In some examples, the second device may be displayed at the default position, and the default position may include a default interface and / or a default address. In the case where the operation object is a window, the default interface may be the current user interface of the second device. In the case where the operation object is a picture, text information, video file, audio file, etc., the default interface may be the user interface of the application for editing, viewing, or processing these files.

[0292] In some examples, the second device may display the application window in a preset manner, and the preset manner is determined according to the type of the second device. The preset manner may be any one of a floating window, a split-screen window, a multi-window, or a full-screen window.

[0293] In some examples, the second device may determine the display manner of the operation object according to the size of its own display screen area. When the size of the display screen of the second device is greater than the first area threshold, the second device may display the user interface including the operation object in the form of a floating window. When the size of the display screen of the second device is less than or equal to the first area threshold, the second device may display the user interface including the operation object in the form of a full-screen window. The operation object may be an application window, a picture, text information, etc.

[0294] In some examples, the response area on the user interface of the second device may also display indication information about the application manner of the second device. For example, the application manner of the second device may include mirroring, stretching, sharing, etc.

[0295] Exemplarily, in response to the user's operation of selecting the operation object on the user interface of the second device, the second device may display one or more pieces of indication information about the application manner of the second device at the position of the response area. In response to the user's operation of dragging the operation object on the user interface of the second device to the display area of one of the application manners, the second device displays the operation object according to the application manner selected by the user.

[0296] In some examples, for different types of operation objects, the second device may display the transmission result of the operation object in different display manners. Exemplarily, for the case where the operation object involves a synchronous transmission task, the second device may explicitly display the transmission result of the operation object. For the case where the operation object does not involve a synchronous transmission task, the second device may non-explicitly display the transmission result of the operation object.

[0297] In some examples, in response to the operation object not being operated within a preset duration, the second device may no longer or stop displaying the operation object. Or rather, in response to the user not operating the operation object within a preset duration, the second device may return to the state before performing the cross-device drag operation.

[0298] S104, display the operation object at the initial position of the first device.

[0299] In some examples, in response to the user stopping dragging or releasing the operation object before dragging the operation object to the trigger area, the first device displays the operation object at the initial position of the operation object.

[0300] In some examples, during the process of the user dragging the operation object, in a scenario where the user needs to select a second device from multiple devices, in response to the user not selecting a second device within a preset duration, the first device displays the operation object at the initial position of the operation object.

[0301] In some examples, during the process of the user dragging the operation object, in a scenario where the user needs to select one of multiple application manners as the application manner of the second device, in response to the user not selecting a second device within a preset duration, the first device displays the operation object at the initial position of the operation object. Here, the identification information of the multiple application manners may be displayed in the trigger area of the first device, or the identification information of the multiple application manners may also be displayed in the response area of the second device.

[0302] In some examples, when the user interface of the second device already displays the operation object, in response to the user not operating the operation object within a preset duration, the second device may return to the state before performing the cross-device drag operation, and the first device displays the operation object at the initial position of the operation object.

[0303] Figure 20 A cross-device collaboration device 2000 provided by an embodiment of the present application. The device 2000 may have the functions of the electronic device in the above method embodiment and may be used to execute the steps performed by the functions of the electronic device in the above method embodiment. This function may be implemented by hardware, or may be implemented by software or hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0304] In a possible implementation, the cross-device collaboration device 2000 may include an acquisition module 2010 and a processing module 2020, and the acquisition module 2010 and the processing module 2020 are coupled to each other.

[0305] In some examples, the acquisition module 2010 may be used to support the first device or the starting device in the foregoing embodiments to acquire user input, such as acquiring a user's drag operation on an operation object, etc.

[0306] The processing module 2020 is used to support the first device or the starting device to perform the processing actions in the above method embodiments, such as displaying an entry control, etc.

[0307] Optionally, the cross-device collaboration device 2000 may further include a storage unit 2030 for storing the program code and data of the cross-device collaboration device 2000.

[0308] In some examples, the acquisition module 2010 may be used to support the second device or the target device in the foregoing embodiments to acquire user input, such as acquiring an operation in which the user selects an application method of the second device on the second device, etc.

[0309] The processing module 2020 is used to support the electronic device to perform the processing actions in the above method embodiments, such as displaying an exit control, etc.

[0310] Optionally, the cross-device collaboration device 2000 may further include a storage unit 2030 for storing the program code and data of the cross-device collaboration device 2000.

[0311] Figure 21 An electronic device 2100 provided in an embodiment of the present application is shown in the figure. The electronic device 2100 includes: at least one processor 2110 and a transceiver 2120. The processor 2110 is coupled to a memory and is used to execute instructions stored in the memory to control the transceiver 2120 to send signals and / or receive signals.

[0312] Optionally, the electronic device 2100 further includes a memory 2130 for storing instructions.

[0313] In some embodiments, the foregoing processor 2110 and the memory 2130 may be integrated into a processing device. The processor 2110 is used to execute program code stored in the memory 2130 to implement the above functions. Specifically, in implementation, the memory 2130 may also be integrated in the processor 2110 or independent of the processor 2110.

[0314] In some embodiments, the transceiver 2120 may include a receiver (or, receiver) and a transmitter (or, transmitter).

[0315] The transceiver 2120 may further include an antenna, and the number of antennas may be one or more. The transceiver 2120 may be a communication interface or an interface circuit.

[0316] When the electronic device 2100 is a chip, the chip includes a transceiver module and a processing module. Among them, the transceiver module may be an input / output circuit or a communication interface; the processing module may be a processor, a microprocessor, or an integrated circuit integrated on the chip.

[0317] This embodiment also provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions run on an electronic device, the electronic device is enabled to execute the above-related method steps to implement the cross-device collaboration method in the above embodiment.

[0318] This embodiment also provides a computer program product. When the computer program product runs on a computer, the computer is enabled to execute the above-related steps to implement the cross-device collaboration method in the above embodiment.

[0319] In addition, an embodiment of the present application also provides a device, which may specifically be a chip, a component, or a module. The device may include a processor and a memory connected to each other. Among them, the memory is used to store computer execution instructions. When the device runs, the processor may execute the computer execution instructions stored in the memory, so that the chip executes the cross-device collaboration method in each of the above method embodiments.

[0320] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professionals can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0321] Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0322] In several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be in electrical, mechanical, or other forms.

[0323] The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0324] In addition, in each embodiment of this application, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0325] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art or a part of this technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of this application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0326] The above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed in this application can easily think of changes or substitutions, which should all be covered within the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

1. A method for cross-device collaboration, applied to a first device, characterized in that, Comprising: Displaying a start interface, the start interface including an operation object; In response to an operation of dragging the operation object to a trigger area and releasing it, sending the operation object to a second device, the trigger area being located on the outer periphery of the display area of the first device; Wherein, the first device and the second device are interconnected, and the second device determines based on one or more of the following: the relative position relationship between the second device and the first device, the placement posture of the second device, or the direction in which the operation object is dragged.

2. The method according to claim 1, wherein Before sending the operation object to the second device in response to the operation of dragging the operation object to the trigger area and releasing it, the method further includes: Displaying one or more shortcuts in the trigger area, the shortcuts being used to indicate the display mode of the operation object; In response to an operation of dragging the operation object to the display area of a first shortcut, sending indication information to the second device, the indication information being used to indicate a first display mode, the first shortcut being used to indicate the first display mode, and the one or more shortcuts including the first shortcut.

3. The method according to claim 1 or 2, characterized in that, The operation of sending the operation object to the second device in response to the operation of dragging the operation object to the trigger area and releasing it includes: Displaying the identification information of the second device in the trigger area; In response to an operation of dragging the operation object to the display area of the identification information of the second device and releasing it, sending the operation object to the second device.

4. The method according to any one of claims 1 to 3, characterized in that, Before sending the operation object to the second device in response to the operation of dragging the operation object to the trigger area and releasing it, the method further includes: In response to an operation of using a first operation mode on a first object, selecting a part of the first object as the operation object; In response to an operation of using a second operation mode on the first object, selecting the first object as the operation object; Wherein, the first operation mode is different from the second operation mode.

5. The method according to any one of claims 1 to 4, characterized in that, Before sending the operation object to the second device in response to the operation of dragging the operation object to the trigger area and releasing it, the method further includes: In response to the distance between the operation object and the edge of the display area of the first device being less than a first threshold, displaying an indication control for indicating the trigger area.

6. The method according to any one of claims 1 to 5, characterized in that, The trigger area is located on the edge of the display area of the first device on the side close to the second device.

7. A method for cross-device collaboration, applied to a first device, characterized in that, Comprising: Displaying a start interface, the start interface including an operation object; In response to a first dragging operation of the operation object along a first dragging direction, displaying a first trigger area on the outer periphery of the start interface corresponding to the first dragging direction; In response to a release operation of the operation object in the first trigger area, sending the operation object to a second device connected to the first device.

8. The method according to claim 7, wherein The method further includes: In response to a second dragging operation of the operation object along a second dragging direction, displaying a second trigger area on the outer periphery of the start interface corresponding to the second dragging direction; In response to a release operation on the operation object in the second trigger area, send the operation object to a third device connected to the first device.

9. The method according to claim 7, characterized in that, The method further includes: In response to an operation of dragging the operation object to the first trigger area, display one or more device identifiers in the first trigger area, where the one or more device identifiers are used to indicate candidate devices, the candidate devices are located on a side of the first device close to the first trigger area, and the candidate devices include the second device; The step of, in response to a release operation on the operation object in the first trigger area, sending the operation object to a second device connected to the first device includes: In response to an operation of dragging the operation object to the display area of the device identifier of the second device and releasing it, send the operation object to the second device.

10. The method according to claim 9, wherein The number of the device identifiers is multiple, and the relative positions of the multiple device identifiers are the same as the relative positions of the multiple candidate devices corresponding to the multiple device identifiers.

11. The method according to claim 9 or 10, characterized in that, The device identifier is further used to indicate the placement posture of the candidate device.

12. A method for cross-device collaboration, characterized in that, It includes: A first device displays a start interface, and the start interface includes an operation object; In response to an operation of dragging the operation object to the trigger area of the first device and releasing it, a second device displays the operation object, the trigger area is located on the outer periphery of the display area of the first device, and the second device is determined according to one or more of the following: the relative position relationship between the second device and the first device, the placement posture of the second device, or the direction in which the operation object is dragged; Wherein, the first device and the second device are connected to each other.

13. The method according to claim 12, wherein The step of, in response to an operation of dragging the operation object to the trigger area of the first device and releasing it, the second device displays the operation object includes: The second device displays the operation object in a response area, and the response area is located on the outer periphery of the display area of the second device.

14. The method according to claim 13, characterized in that, The step that the second device displays the operation object in the response area includes: The second device displays the operation object at an exit position of the response area, and the exit position is determined according to an entrance position, and the entrance position is the position where the operation object is released in the trigger area.

15. The method according to claim 13 or 14, characterized in that, One or more shortcuts are also displayed in the response area, and the shortcuts are used to indicate the display mode of the operation object. The method further includes: In response to an operation of dragging the operation object to the display area of a second shortcut, the second device displays the operation object in a second display mode, and the second shortcut is used to indicate the second display mode, and the one or more shortcuts include the second shortcut.

16. The method according to any one of claims 12 to 15, characterized in that The method further includes: When the second device displays the operation object, in response to the operation object not being operated within a preset duration, the second device stops displaying the operation object.

17. The method according to any one of claims 12 to 16, characterized in that, The operation object is an application window. The step of, in response to an operation of dragging the operation object to the trigger area of the first device and releasing it, the second device displays the operation object includes: The second device displays the application window in a preset manner, which is determined according to the type of the second device, and the preset manner includes any one of a floating window, a split-screen window, a multi-window or a full-screen window.

18. The method according to any one of claims 12 to 17, characterized in that In response to an operation of dragging the operation object to the trigger area of the first device and releasing it, the second device displays the operation object, including: The second device displays a user interface of a target application, and the target application is used to edit the operation object.

19. An electronic device, characterized in that, It includes a processor and a memory, the memory is used to store program instructions, and the processor is used to call the program instructions to execute the method described in any one of claims 1 to 6 or any one of claims 7 to 11.

20. A computer-readable storage medium, characterized in that, A computer program is stored thereon, and when the computer program is executed by a computer, it enables the implementation of the method described in any one of claims 1 to 6 or any one of claims 7 to 11.

Images