Cross-device collaboration method and electronic device

Abstract

The present application provides a cross-device collaboration method and an electronic device. A user can drag an operation object on a starting device to a trigger area on the starting device, such that cross-device dragging of the operation object can be triggered, and a target device can correspondingly display the operation object. To trigger cross-device dragging, the user does not need to continuously drag the operation object from a user interface of the starting device to a user interface of the target device, and the cross-device dragging performed between the starting device and the target device is less affected by hardware conditions of the starting device, such that the efficiency of cross-device dragging between two devices is higher, and the capability of collaboratively processing tasks between the two devices is higher, thus achieving better use experience for users.

Description

Cross-device collaboration method and electronic device

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on January 31, 2024, with application number 202410144514.8 and invention name “Method and electronic device for cross-device collaboration”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of terminal device software, and in particular, to a method and electronic device for cross-device collaboration. Background Art

[0003] With the development of electronic technology and mobile Internet, the same user can often own multiple electronic devices at the same time, such as mobile phones, tablets, personal computers, smart home devices, etc. There are more and more scenarios such as collaborative office and smart home that require multiple electronic devices to be connected and used together.

[0004] Different devices typically possess different capabilities, such as communication, data processing, and image processing. Compared to individual devices functioning independently, collaborative devices can more fully leverage the diverse capabilities of different electronic devices, improving their ability to handle diverse tasks in different scenarios and enhancing the user experience.

[0005] How to improve the efficiency of collaborative task processing among multiple devices is an issue worth considering. Summary of the Invention

[0006] This application provides a method for cross-device collaboration. A user can drag an operating object located on a first device to a trigger area on the first device to trigger the dragging of the operating object between the first and second devices. This allows the operating object to be dragged across the two devices without requiring a long drag stroke. This increases the efficiency of data and capability sharing between the first and second devices, and enhances the ability of the two devices to collaboratively process tasks.

[0007] In a first aspect, a method for cross-device collaboration is provided, including: a first device displays a starting interface, the starting interface includes an operation object; in response to the operation object being dragged to a trigger area of ​​the first device and released, the second device displays the operation object, and the trigger area is located outside the display area of ​​the first device; wherein the first device and the second device are interconnected, and the second device is determined based on one or more of the following: the relative position relationship between the second device and the first device, the placement posture of the second device, or the direction in which the operation object is dragged.

[0008] In a possible implementation, the first device and the second device may be communicated with each other in a wireless manner and / or a wired manner.

[0009] In one possible implementation, the operation object may be one or more interface elements or the content of interface elements on the starting interface. For example, the operation object may include one or more of the following: text, images, code, files, interface components, and application windows. Interface components may include buttons, text boxes, sliders, and icons. Application windows may include full-screen windows, split-screen windows, or floating windows.

[0010] In some scenarios, the trigger area may also be called the drag sensing area or the cross-device drag operation sensing area.

[0011] In a possible implementation, the trigger area may include multiple sub-areas on the periphery of the display area of ​​the first device.

[0012] In this technical solution, the first device can be provided with a sensing area for triggering cross-device dragging. The user can drag the operation object from the first device to the sensing area to trigger the cross-device dragging of the operation object without continuously dragging the operation object from the first device to the second device. The cross-device dragging between the first device and the second device is more efficient, which is conducive to improving the efficiency of collaborative task processing between the two settings.

[0013] In combination with the first aspect, in certain implementations of the first aspect, in response to the operation of dragging the operation object to the trigger area of ​​the first device and releasing it, the second device displays the operation object, including: the second device displays the operation object in the response area, and the response area is located outside the display area of ​​the second device.

[0014] In a possible implementation, the response area is located at an edge of a side of the display area of ​​the second device close to the first device.

[0015] The operation object released from the periphery of the display area of ​​the first device appears on the periphery of the display area of ​​the second device when cross-device dragging is triggered. This technical solution is more in line with the user's intuition of performing cross-device dragging. On the one hand, it is conducive to improving the efficiency of cross-device dragging between the two devices. On the other hand, compared with displaying the operation object in the middle area of ​​the display area of ​​the second device, this solution is conducive to reducing the interference of cross-device dragging on the tasks being executed on the second device, which is conducive to improving the user experience.

[0016] In combination with the first aspect, in certain implementations of the first aspect, the second device displays the operation object in the response area, including: the second device displays the operation object at the exit position of the response area, the exit position is determined according to the entrance position, and the entrance position is the position where the operation object is released in the trigger area.

[0017] In one possible implementation, the distances between the outlet and the upper and lower edges of the second device are L1 and L2 respectively, and the distances between the inlet and the upper and lower edges of the first device are L3 and L4 respectively, then: L1:L2=L3:L4.

[0018] In a possible implementation, the first device sends information about the entrance location to the second device.

[0019] The entry position of the operation triggered during the cross-device drag process is used to determine the exit position where the operation object appears on the second device. This technical solution is more in line with the user's intuition of performing cross-device drag, which is conducive to improving the efficiency of performing cross-device drag between two devices and improving the user experience.

[0020] In combination with the first aspect, in certain implementations of the first aspect, the response area also displays one or more shortcuts, and the shortcuts are used to indicate the display mode of the operation object. The method also includes: in response to the operation of dragging the operation object to the display area of ​​the second shortcut, the second device displays the operation object in a second display mode, the second shortcut is used to indicate the second display mode, and the one or more shortcuts include the second shortcut.

[0021] In some scenarios, the shortcut displayed in the response area may also be referred to as indication information of the application mode of the second device.

[0022] In some scenarios, the display mode of the operation object can also be understood as the application mode of the second device.

[0023] In a possible implementation, the application mode of the second device includes: mirroring, extension, sharing, etc., and the second application mode may be any one of these application modes.

[0024] During each cross-device drag operation, the user can reselect the application mode of the second device, which is beneficial to improving the efficiency of the collaborative work between the first device and the second device to a certain extent, and enhancing the user's experience of cross-device drag operations. With reference to the first aspect, in certain implementations of the first aspect, the second device displays the operation object in the response area, including: the second device highlights the operation object in the response area.

[0025] In one possible implementation, the second device can display the operation object in a highlighted color, the second device can also display the operation object with animation effects (such as shaking, deformation, etc.), the second device can also enlarge the display of the operation object, and the second device can also hover the display of the operation object at the position of the response area.

[0026] By highlighting the operation object of cross-device dragging to remind the user to handle it in time, the implementation of this technical solution is conducive to improving the success rate of dragging the operation object across devices, and to a certain extent is conducive to improving the efficiency of collaborative work between the first device and the second device, and enhancing the user's experience of cross-device dragging operations.

[0027] In combination with the first aspect, in some implementations of the first aspect, the method further includes: when the second device displays the operation object,

[0028] In response to the operation object not being operated within a preset time period, the second device stops displaying the operation object.

[0029] In some scenarios, the above solution can also be understood as: in response to the user not operating the operation object within a preset time period, the second device can be restored to the state before the cross-device drag operation is performed.

[0030] This technical solution sets a time threshold to control the user's response to the operation object of the cross-device drag operation. When the time threshold is exceeded, the first device and the second device can stop executing the cross-device drag task. The implementation of this technical solution is conducive to reducing the impact of cross-device drag on other tasks that the first device and the second device are currently executing or about to execute, and is conducive to improving the utilization efficiency of software resources and hardware resources of the first device and the second device.

[0031] In combination with the first aspect, in certain implementations of the first aspect, the operation object is an application window, and in response to the operation of dragging the operation object to the trigger area of ​​the first device and releasing it, the second device displays the operation object, including: the second device displays the application window in a preset manner, the preset manner is determined according to the type of the second device, and the preset manner includes: any one of a floating window, a split-screen window, a multi-window or a full-screen window.

[0032] In the case where the operation object is an application window, the second device can determine the display method of the operation object of the cross-device drag operation based on the size of its own display area, which is conducive to improving the display effect of the operation object on the second device, and improving the user experience of the cross-device drag operation. To a certain extent, it is also conducive to improving the ability of the first device and the second device to work together.

[0033] In combination with the first aspect, in certain implementations of the first aspect, in response to the operation of dragging the operation object to the trigger area of ​​the first device and releasing it, the operation object is displayed on the second device, including: the second device displays the user interface of the target application, and the target application is used to edit the operation object.

[0034] In response to the user's cross-device drag operation, the second device can directly open the application window for processing the operation object and display the user interface for editing or processing the operation object. The implementation of this technical solution is conducive to improving the efficiency of executing cross-device drag tasks between the two devices, improving the user experience, and to a certain extent, it is also conducive to improving the ability of the first device and the second device to work together.

[0035] In combination with the first aspect, in some implementations of the first aspect, the trigger area also displays one or more shortcuts, the shortcuts are used to indicate the display mode of the operation object, and in response to the operation object being dragged to the trigger area of ​​the first device and released, the second device displays the operation object, including: in response to the operation object being dragged to the display area of ​​the first shortcut, the second device displays the operation object in the first display mode, the first shortcut is used to indicate the first display mode, and the one or more shortcuts include the first shortcut.

[0036] In some scenarios, the shortcut displayed in the trigger area can also be referred to as indication information of the application mode of the second device.

[0037] In some scenarios, the display mode of the operation object can also be understood as the application mode of the second device.

[0038] In a possible implementation, the application mode of the second device includes: mirroring, extension, sharing, etc., and the second application mode may be any one of these application modes.

[0039] During each cross-device drag operation, the user can reselect the application mode of the second device, which is beneficial to improve the efficiency of the collaborative work between the first device and the second device to a certain extent, and enhance the user's experience of cross-device drag operations.

[0040] In conjunction with the first aspect, in certain implementations of the first aspect, in response to an operation of dragging an operation object to a trigger area of ​​the first device and releasing the operation object, the second device displaying the operation object includes:

[0041] In response to an operation of using a first operating body to act on a first object in a first operating manner, the second device displays a target interface including the first object, or, in response to an operation of using a second operating body to act on a second object in a second operating manner, the second device displays a target interface including the second operating object; wherein the operating objects include the first object and the second object, the first object is different from the second object, the first operating body is different from the second operating body, and / or the first operating manner is different from the second operating manner.

[0042] In one possible implementation, in response to an operation performed on a first object using a first operation mode, a portion of the first object is selected as an operation object; in response to an operation performed on the first object using a second operation mode, the first object is selected as an operation object; wherein the first operation mode is different from the second operation mode.

[0043] In a possible implementation, the first operating body or the second operating body may include a finger, a stylus, or a mouse, etc. The first operation mode or the second operation mode may include single-finger operation, two-finger operation, palm operation, long press operation, hard press operation, or knuckle operation, etc.

[0044] The implementation of this technical solution, which triggers cross-device dragging of different types of operation objects through different operating bodies and / or operation methods, is conducive to improving the applicability of the cross-device dragging method provided in this application in different scenarios, improving the efficiency of multiple devices in executing cross-device dragging tasks, and improving the ability of the first device and the second device to work together.

[0045] In combination with the first aspect, in certain implementations of the first aspect, before the second device displays the operation object in response to the operation of dragging the operation object to the trigger area of ​​the first device and releasing it, the method also includes: in response to the distance between the operation object and the edge of the display area of ​​the first device being less than a first threshold, the first device displays an indicator control, and the indicator control is used to indicate the trigger area.

[0046] In this technical solution, an indicator control is used to indicate the recognition area that triggers cross-device dragging. The position of the recognition area is easier for the user to perceive, the success rate of the user performing the cross-device dragging operation is higher, and the user's cross-device dragging operation experience is better.

[0047] In combination with the first aspect, in some implementations of the first aspect, the trigger area is located at an edge of a display area of ​​the first device on a side close to the second device.

[0048] The trigger area is set at a position where the display area of ​​the first device is close to the edge of the second device. This technical solution is more in line with the user's intuition of performing cross-device dragging, which is conducive to improving the efficiency of performing cross-device dragging between the two devices and improving the user experience.

[0049] In combination with the first aspect, in certain implementations of the first aspect, before the second device displays the operation object in response to the operation of dragging the operation object to the trigger area of ​​the first device and releasing it, the method also includes: the first device displays the identification information of the second device in the trigger area; in response to the user dragging the operation object to the display area of ​​the identification information of the second device, the second device displays the operation object.

[0050] This technical solution combines multiple factors to determine the second device for the cross-device drag operation, which is conducive to making the selected second device more suitable for performing cross-device drag operations and more suitable for completing collaborative operations with the first device. The implementation of this technical solution is conducive to improving the ability of the first device and the second device to work collaboratively.

[0051] For the relevant explanations and descriptions of the beneficial effects in the following technical solutions, please refer to the relevant content in the first aspect in the previous text, and will not be repeated below.

[0052] In a second aspect, a method for cross-device collaboration is provided, which is applied to a first device, including: displaying a starting interface, the starting interface including an operation object; in response to the operation of dragging the operation object to a trigger area and releasing it, sending the operation object to a second device, and the trigger area is located on the periphery of the display area of ​​the first device; wherein the first device and the second device are interconnected, and the second device is determined based on one or more of the following: the relative position relationship between the second device and the first device, the placement posture of the second device, or the direction in which the operation object is dragged.

[0053] In combination with the second aspect, in certain implementations of the second aspect, in response to the operation of dragging the operation object to the trigger area and releasing it, before sending the operation object to the second device, one or more shortcuts are displayed in the trigger area, and the shortcuts are used to indicate the display mode of the operation object; in response to the operation of dragging the operation object to the display area of ​​the first shortcut, indication information is sent to the second device, the indication information is used to indicate the first display mode, the first shortcut is used to indicate the first display mode, and the one or more shortcuts include the first shortcut.

[0054] In combination with the second aspect, in certain implementations of the second aspect, in response to the operation of dragging the operation object to the trigger area and releasing it, sending the operation object to the second device, including: displaying the identification information of the second device in the trigger area; in response to the operation of dragging the operation object to the display area of ​​the identification information of the second device and releasing it, sending the operation object to the second device.

[0055] In combination with the second aspect, in certain implementations of the second aspect, in response to the operation of dragging the operation object to the trigger area and releasing it, before sending the operation object to the second device, in response to the operation of acting on the first object using the first operation method, a part of the first object is selected as the operation object; in response to the operation of acting on the first object using the second operation method, the first object is selected as the operation object; wherein, the first operation method is different from the second operation method.

[0056] In combination with the second aspect, in certain implementations of the second aspect, in response to the operation of dragging the operation object to the trigger area and releasing it, before sending the operation object to the second device, in response to the distance between the operation object and the edge of the display area of ​​the first device being less than a first threshold, an indicator control is displayed, and the indicator control is used to indicate the trigger area.

[0057] In combination with the second aspect, in some implementations of the second aspect, the trigger area is located at an edge of a display area of ​​the first device on a side close to the second device.

[0058] In a third aspect, a method for cross-device coordination is provided, which is applied to a first device, including: displaying a starting interface, the starting interface including an operation object; in response to a first drag operation on the operation object along a first drag direction, displaying a first trigger area on the periphery of the starting interface corresponding to the first drag direction; in response to a release operation on the operation object in the first trigger area, sending the operation object to a second device connected to the first device.

[0059] In a possible implementation, the connection direction between the second device and the first device is parallel to the dragging direction.

[0060] It should be understood that the “parallel” here can be understood as approximately parallel or substantially parallel, etc.

[0061] In combination with the third aspect, in certain implementations of the third aspect, the method also includes: in response to a second drag operation on the operation object along the second drag direction, displaying a second trigger area on the periphery of the starting interface corresponding to the second drag direction; in response to a release operation on the operation object in the second trigger area, sending the operation object to a third device connected to the first device.

[0062] In combination with the third aspect, in certain implementations of the third aspect, the method also includes: in response to the operation of dragging the operation object to the first trigger area, displaying one or more device identifiers in the first trigger area, the one or more device identifiers are used to indicate candidate devices, the candidate devices are located on the side of the first device close to the first trigger area, and the candidate devices include the second device; in response to the release operation of the operation object in the first trigger area, sending the operation object to the second device connected to the first device, including: in response to the operation of dragging the operation object to the display area of ​​the device identifier of the second device and releasing it, sending the operation object to the second device.

[0063] In combination with the third aspect, in certain implementations of the third aspect, there are multiple device identifiers, and the relative positions of the multiple device identifiers are the same as the relative positions of the multiple candidate devices corresponding to the multiple device identifiers.

[0064] In combination with the third aspect, in certain implementations of the third aspect, the device identifier is further used to indicate the placement posture of the candidate device.

[0065] In a fourth aspect, a cross-device collaborative device is provided, which includes a first module and a second module, the first module being used to: display a starting interface, the starting interface including an operation object; the second module being used to: display the operation object in response to the operation of dragging the operation object to the trigger area of ​​the first module and releasing it, and the trigger area being located on the periphery of the display area of ​​the first module; wherein the first module and the second module are interconnected, and the second module is determined based on one or more of the following: the relative position relationship between the second module and the first module, the placement posture of the second module, or the direction in which the operation object is dragged.

[0066] In combination with the fourth aspect, in certain implementations of the fourth aspect, the second module is specifically configured to: display the operation object in a response area, and the response area is located outside the display area of ​​the second module.

[0067] In combination with the fourth aspect, in certain implementations of the fourth aspect, the second module is specifically used to: display the operation object at the exit position of the response area, the exit position is determined according to the entry position, and the entry position is the position where the operation object is released in the trigger area.

[0068] In combination with the fourth aspect, in certain implementations of the fourth aspect, the response area also displays one or more shortcuts, which are used to indicate the display mode of the operation object. The second module is also used to: in response to the operation of dragging the operation object to the display area of ​​the second shortcut, display the operation object in a second display mode, the second shortcut is used to indicate the second display mode, and the one or more shortcuts include the second shortcut.

[0069] In combination with the fourth aspect, in some implementations of the fourth aspect, the second module is further used to: when the second module displays the operation object, in response to the operation object not being operated within a preset time period, stop displaying the operation object.

[0070] In combination with the fourth aspect, in certain implementations of the fourth aspect, the operation object is an application window, and the second module is specifically used to: display the application window in a preset manner, the preset manner is determined according to the type of the second module, and the preset manner includes: any one of a floating window, a split-screen window, multiple windows or a full-screen window.

[0071] In combination with the fourth aspect, in certain implementations of the fourth aspect, the second module is specifically used to: display a user interface of a target application, where the target application is used to edit the operation object.

[0072] In combination with the fourth aspect, in certain implementations of the fourth aspect, the trigger area also displays one or more shortcuts, which are used to indicate the display mode of the operation object. The second module is specifically used to: in response to the operation of dragging the operation object to the display area of ​​the first shortcut, display the operation object in a first display mode, the first shortcut is used to indicate the first display mode, and the one or more shortcuts include the first shortcut.

[0073] In combination with the fourth aspect, in certain implementations of the fourth aspect, the second module is specifically used to: in response to an operation of using a first operating body to act on a first object in a first operating manner, display a target interface including the first object, or, in response to an operation of using a second operating body to act on a second object in a second operating manner, display a target interface including the second operating object; wherein the operating objects include a first object and a second object, the first object is different from the second object, the first operating body is different from the second operating body, and / or the first operating manner is different from the second operating manner.

[0074] In combination with the fourth aspect, in certain implementations of the fourth aspect, the first module is further used to: in response to the distance between the operating object and the edge of the display area of ​​the first module being less than a first threshold, display an indicator control, where the indicator control is used to indicate the trigger area.

[0075] In combination with the fourth aspect, in certain implementations of the fourth aspect, the trigger area is located at an edge of a display area of ​​the first module on a side close to the second module.

[0076] In combination with the fourth aspect, in certain implementations of the fourth aspect, in response to the operation object being dragged to the trigger area of ​​the first module and released, before the second module displays the operation object, the first module is used to: display the identification information of the second device in the trigger area; in response to the user dragging the operation object to the display area of ​​the identification information of the second module, the second module is used to: display the operation object.

[0077] In a fifth aspect, a first device is provided, comprising a processor and a memory, the memory being configured to store program instructions, the processor being configured to: display a starting interface including an operation object; and, in response to dragging and releasing the operation object to a trigger area, sending the operation object to a second device, the trigger area being located outside a display area of ​​the first device; wherein the first and second devices are interconnected, and the second device determines the direction of the operation object based on one or more of: a relative positional relationship between the second device and the first device, a placement posture of the second device, or a direction in which the operation object is dragged.

[0078] In combination with the fifth aspect, in certain implementations of the fifth aspect, the processor is further used to: in response to a distance between the operating object and the edge of the display area of ​​the first device being less than a first threshold, display an indicator control, where the indicator control is used to indicate the trigger area.

[0079] In combination with the fifth aspect, in some implementations of the fifth aspect, the trigger area is located at an edge of a display area of ​​the first device on a side close to the second device.

[0080] In combination with the fifth aspect, in certain implementations of the fifth aspect, the trigger area further displays one or more shortcuts, and the shortcuts are used to indicate the display method of the operation object.

[0081] In a possible implementation, the processor is further configured to: send first indication information, where the first indication information is used to indicate a display mode of the operation object.

[0082] In combination with the fifth aspect, in certain implementations of the fifth aspect, the processor is further used to: send second indication information, where the second indication information is used to indicate an entry position, where the entry position is a position where the operating object is released in the trigger area.

[0083] In combination with the fifth aspect, in certain implementations of the fifth aspect, in response to the operation of dragging the operation object to the trigger area of ​​the first device and releasing it, the processor is further used to: display identification information of the second device in the trigger area.

[0084] In a sixth aspect, a second device is provided, which includes a processor and a memory, the memory is used to store program instructions, and the processor is used to: receive an operation object and display the operation object.

[0085] In combination with the sixth aspect, in certain implementations of the sixth aspect, the processor is specifically used to: display the operation object in a response area, and the response area is located outside the display area of ​​the second device.

[0086] In combination with the sixth aspect, in certain implementations of the sixth aspect, the processor is specifically used to: display the operation object at the exit position of the response area, the exit position is determined according to the entry position, and the entry position is the position where the operation object is released in the trigger area.

[0087] In combination with the sixth aspect, in certain implementations of the sixth aspect, the response area also displays one or more shortcuts, and the shortcuts are used to indicate the display mode of the operation object. The processor is also used to: in response to the operation of dragging the operation object to the display area of ​​the second shortcut, display the operation object in the second display mode, and the second shortcut is used to indicate the second display mode, and the one or more shortcuts include the second shortcut.

[0088] In combination with the sixth aspect, in certain implementations of the sixth aspect, the processor is further used to: when the second device displays the operation object, in response to the operation object not being operated within a preset time period, stop displaying the operation object.

[0089] In combination with the sixth aspect, in certain implementations of the sixth aspect, the operation object is an application window, and the processor is specifically used to: display the application window in a preset manner, the preset manner is determined according to the type of the second device, and the preset manner includes: any one of a floating window, a split-screen window, a multi-window or a full-screen window.

[0090] In combination with the sixth aspect, in certain implementations of the sixth aspect, the processor is specifically used to: display a user interface of a target application, where the target application is used to edit the operation object.

[0091] In conjunction with the sixth aspect, in certain implementations of the sixth aspect, the processor is specifically configured to: receive first indication information, where the first indication information is used to indicate a display mode for the operation object. The processor is further configured to: display the operation object in a first display mode according to the first indication information, where the first shortcut is used to indicate the first display mode, and the one or more shortcuts include the first shortcut.

[0092] In combination with the sixth aspect, in certain implementations of the sixth aspect, the processor is specifically used to: in response to an operation of using a first operating body to act on a first object in a first operating manner, display a target interface including the first object, or, in response to an operation of using a second operating body to act on a second object in a second operating manner, display a target interface including the second operating object; wherein the operating objects include a first object and a second object, the first object is different from the second object, the first operating body is different from the second operating body, and / or the first operating manner is different from the second operating manner.

[0093] In the seventh aspect, a computer program product is provided, which includes computer program code, and when the computer program code is run on a computer, the method in the second aspect and any possible implementation thereof is executed, or the method in the third aspect and any possible implementation thereof is executed.

[0094] In an eighth aspect, a computer-readable storage medium is provided, which stores a computer program code. When the computer program code runs on a computer, the method in the second aspect and any possible implementation thereof is executed, or the method in the third aspect and any possible implementation thereof is executed.

[0095] In the ninth aspect, a chip is provided, comprising a processor for reading instructions stored in a memory, wherein when the processor executes the instructions, the chip implements the method in the second aspect and any possible implementation thereof, or the chip implements the method in the third aspect and any possible implementation thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0096] FIG1 is a schematic diagram of a hardware architecture of an electronic device applicable to an embodiment of the present application.

[0097] FIG2 is a schematic diagram of a software architecture of an electronic device applicable to an embodiment of the present application.

[0098] FIG3 is a process diagram of a cross-device drag operation provided by an embodiment of the present application.

[0099] FIG4 is a process diagram of another cross-device drag operation provided by an embodiment of the present application.

[0100] FIG5 is a schematic diagram of a drag sensing area of ​​an initiating device provided in an embodiment of the present application.

[0101] FIG6 is a schematic diagram of a method for selecting a target device provided in an embodiment of the present application.

[0102] FIG7 is a schematic diagram of another target device selection method provided in an embodiment of the present application.

[0103] FIG8 is a schematic diagram of another method for selecting a target device provided in an embodiment of the present application.

[0104] FIG9 is a schematic diagram of another method for selecting a target device provided in an embodiment of the present application.

[0105] FIG10 is a schematic diagram of an operation object selection method provided in an embodiment of the present application.

[0106] FIG11 is a schematic diagram of a method for selecting an application mode of a target device provided in an embodiment of the present application.

[0107] FIG12 is a process diagram of another cross-device drag operation provided in an embodiment of the present application.

[0108] 13 to 18 are schematic diagrams of a method for a target device to process an operation object of a cross-device drag operation according to an embodiment of the present application.

[0109] FIG19 is a schematic diagram of a method for collaborative work between devices provided in an embodiment of the present application.

[0110] FIG20 is a schematic diagram of a cross-device collaboration apparatus provided in an embodiment of the present application.

[0111] FIG21 is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0112] The technical solution in this application will be described below with reference to the accompanying drawings.

[0113] The terms used in the following embodiments are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification of this application and the appended claims, the singular expressions "a", "an", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear contrary indication in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one, two or more. The term "and / or" is used to describe the association relationship of associated objects, indicating that three relationships can exist; for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship.

[0114] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0115] 1 shows a schematic structural diagram of an electronic device 100. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display 194, and a subscriber identification module (SIM) card interface 195. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0116] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0117] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0118] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0119] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0120] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0121] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display 194 and the camera 193. MIPI interfaces include the camera serial interface (CSI) and the display serial interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the camera function of the electronic device 100. The processor 110 and the display 194 communicate via the DSI interface to implement the display function of the electronic device 100.

[0122] The USB interface 130 is an interface that complies with USB standards and may be a Mini USB interface, a Micro USB interface, a USB Type-C interface, or the like. The USB interface 130 can be used to connect a charger to charge the electronic device 100, or to transfer data between the electronic device 100 and peripheral devices. It can also be used to connect headphones to play audio. This interface can also be used to connect other electronic devices, such as augmented reality devices.

[0123] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0124] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0125] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0126] Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0127] Electronic device 100 can implement a camera function using an ISP, camera 193, a video codec, a GPU, a display 194, and an application processor. The ISP processes data fed back by camera 193. Camera 193 is used to capture still images or video. The digital signal processor processes digital signals, and can process not only digital image signals but also other digital signals. The video codec compresses or decompresses digital video.

[0128] NPU is a neural-network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn.

[0129] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function.

[0130] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area can store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0131] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0132] The buttons 190 include a power button, a volume button, and the like. The buttons 190 may be mechanical buttons or touch buttons. The electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100.

[0133] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts and can also be used for touch vibration feedback.

[0134] The indicator 192 may be an indicator light, which may be used to indicate the charging status, power level changes, messages, missed calls, notifications, etc.

[0135] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. In the embodiment of the present application, the Android system with a layered architecture is used as an example to illustrate the software structure of the electronic device 100.

[0136] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0137] Figure 2 is a software structure diagram of the electronic device 100 according to an embodiment of the present application. The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers: the application (app) layer, the application framework layer, the Android runtime (Android runtime) and system libraries, and the kernel layer. The application layer may include a series of application packages.

[0138] As shown in FIG2 , the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and short message.

[0139] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0140] As shown in FIG2 , the application framework layer may include a window manager, an activity manager, a package manager, a resource manager, a view system, a telephony manager, a notification manager, and the like.

[0141] Resource Manager, also known as Resource Management Service (RMS), provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0142] A window manager, also known as a window management service (WMS), manages windowed programs. It can determine the display size, determine whether a status bar is present, lock the screen, and take screenshots.

[0143] The activity manager, also known as the activity manager service (AMS), manages all application processes in the system.

[0144] The package manager, also known as the package management service (PMS), is responsible for functions such as application installation and uninstallation, component query and matching, and permission management.

[0145] The view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon might include a view for displaying text and a view for displaying images.

[0146] The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically without user interaction. For example, the Notification Manager is used to notify users of completed downloads and message reminders. The Notification Manager can also display notifications in the top status bar of the system as icons or scrolling text, such as notifications from background applications, or as dialog windows on the screen. Examples include text messages in the status bar, beeps, vibrations on electronic devices, and flashing indicator lights.

[0147] The system library can include multiple functional modules, such as a surface manager, media libraries, a 3D graphics processing library (such as OpenGL ES), and a 2D graphics engine (such as SGL).

[0148] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

[0149] The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. The media library can support a variety of audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0150] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0151] A 2D graphics engine is a drawing engine for 2D drawings.

[0152] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver, and sensor driver.

[0153] FIG3 is a schematic diagram illustrating a cross-device drag operation process provided by an embodiment of the present application, involving an initiating device 10 and a target device 20, which may be connected to each other. At a first moment, an operation object 30 may be displayed on the user interface of the initiating device 10. In response to a user continuously dragging the operation object from the user interface of the initiating device 10 to the target device 20, at a second moment, the operation object 30 may be displayed on the user interface of the target device 20.

[0154] For example, the initiating device 10 may be connected to a mouse, and the user may use the mouse to select and continuously drag the operation object 30 to complete the operation of dragging the operation object 30 from the user interface of the initiating device 10 to the user interface of the target device 20. For another example, the initiating device 10 may include a touchpad, and the user may use the touchpad to select and continuously drag the operation object 30 to complete the cross-device drag operation of the operation object 30 from the initiating device 10 to the target device 20.

[0155] In the above-mentioned cross-device drag operation scenario, if the initiating device 10 is not connected to a mouse or does not include a touchpad, or if the initiating device 10 includes a touchpad but the operable area of ​​the touchpad is too small to meet the drag range requirements of the cross-device drag, it is difficult for the user to drag the operation object 30 from the initiating device 10 to the target device 20. The efficiency of the collaboration between the initiating device 10 and the target device 20 is low, and the user experience of cross-device dragging is poor.

[0156] In order to improve the efficiency of collaborative task processing between multiple devices and enhance the user experience of dragging and dropping operations across devices, this application provides a method for cross-device collaboration, which is introduced as follows.

[0157] FIG4 is a schematic diagram of another cross-device drag operation process provided by the present application, still taking the initiating device 10 and the target device 20 as an example, the initiating device 10 and the target device 20 may be connected to each other.

[0158] An entry control 40A may be displayed on the user interface of the initiating device 10. This entry control 40A may be used to indicate an entry area that triggers a cross-device drag operation. In some scenarios, this entry area may also be referred to as a trigger area. Taking the operation object 30 on the user interface of the initiating device 10 as an example, in response to a user dragging the operation object 30 to the entry area, the operation object 30 may be displayed on the user interface of the target device 20.

[0159] In the scenario of cross-device dragging, the operation object 30 located at the entrance area of ​​the starting device 10 can be directly displayed on the target device 20. The user can drag the operation object 30 across devices without having to perform a continuous drag operation from the starting device 10 to the target device 20. The cross-device drag operation is not limited to the hardware of the starting device 10. The implementation of this technical solution is conducive to improving the efficiency of collaborative work between the starting device 10 and the target device 20, and is conducive to improving the user's experience of cross-device drag operations.

[0160] In some examples, the initiating device 10 and the target device 20 can establish a connection via a wired or wireless method. Based on the established connection, the initiating device 10 and the target device 20 can be used together, that is, the collaborative work described above. The initiating device 10 and the target device 20 can use a variety of wireless communication protocols for wireless connection. For example, the wireless communication protocol can be a wireless fidelity (Wi-Fi) protocol, a Bluetooth protocol, a ZigBee protocol, a near field communication (NFC) protocol, various cellular network protocols, etc., which are not limited in this application.

[0161] In some examples, the starting device 10 and / or the target device 20 can be a mobile phone, a tablet computer, a handheld computer, a PC, a cellular phone, a personal digital assistant (PDA), a wearable device (such as a smart watch), a smart home device (such as a TV), an in-vehicle computer, a game console, and an augmented reality (AR) or virtual reality (VR) device, etc., and the specific device form is not limited. For an introduction to the hardware architecture and software architecture of the starting device 10 and / or the target device 20, reference can be made to the relevant description of the electronic device 100 in the previous text. The device forms of the starting device 10 and the target device 20 can be the same or different, and this application does not limit this. For example, the starting device 10 and the target device 20 can both be touch screen devices or both be non-touch screen devices.

[0162] The entry area of ​​the starting device 10 can also be called the trigger area or drag perception area for cross-device dragging. In response to the user's dragging operation on the operating object on the user interface of the starting device 10, an entry control 40A can be displayed on the user interface of the starting device 10, and the entry control 40A can be used to indicate the aforementioned drag perception area. Similarly, the target device 20 can also be provided with a drag perception area. In response to the user's dragging operation on the operating object on the user interface of the target device 20, a control similar to the aforementioned entry control 40A can be displayed on the user interface of the target device 20, and the control can be used to indicate the drag perception area on the target device 20. The following introduces the drag perception area by taking the process of dragging the operating object on the starting device 10 to the target device 20 as an example.

[0163] In some examples, the drag-sensing area may be located at the periphery of the display area of ​​the initiating device 10, or in other words, the aforementioned entry control 40A may be displayed at the periphery of the display area of ​​the initiating device 10. For example, as shown in FIG5 , the entry control 40A may include sub-controls 41A, 41B, 40C, and 40D. The drag-sensing area of ​​the initiating device 10 may include a first sub-area, a second sub-area, a third sub-area, and a fourth sub-area, respectively located at the right edge, top edge, left edge, and bottom edge of the display area of ​​the initiating device 10. The four sub-controls of the entry control 40A may be used to indicate the four sub-areas of the entry area, namely, sub-control 41A may be used to indicate the first sub-area, sub-control 41B may be used to indicate the second sub-area, sub-control 41C may be used to indicate the third sub-area, and sub-control 41D may be used to indicate the fourth sub-area.

[0164] In some examples, in response to a user drag operation on the user interface of the initiating device 10 , the initiating device 10 may simultaneously display the four sub-controls.

[0165] In some examples, in response to a user dragging operation in the user interface of initiating device 10, initiating device 10 may display a sub-control in the drag-sensing area corresponding to the direction of the user's dragging operation (or drag direction). For example, if an operation object is located in a non-drag-sensing area of ​​the display area of ​​initiating device 10, in response to the user dragging the operation object to the right, initiating device 10 may display sub-control 41A located at the right edge of the display area. In response to the user dragging the operation object downward, initiating device 10 may display sub-control 41D located at the lower edge of the display area.

[0166] In one possible implementation, the initiating device 10 may determine the direction of the user's drag operation based on capacitance information of its own display screen. For example, the initiating device 10 may obtain changes in capacitance values ​​in different areas of the display screen during the user's dragging process, determine the path of the user's drag operation based on the changes in capacitance values ​​in different areas, and thus determine the direction of the user's drag operation.

[0167] In some examples, in response to a user's drag operation on the user interface of initiating device 10, initiating device 10 may display a sub-control corresponding to the position of target device 20 in the drag-aware area based on the relative spatial positional relationship between target device 20 and initiating device 10. For example, if target device 20 is located to the right of initiating device 10, in response to the user's drag operation, initiating device 10 may display sub-control 41A located at the right edge of the display area. If target device 20 is located above initiating device 10, in response to the user's drag operation, initiating device 10 may display sub-control 41B located at the upper edge of the display area.

[0168] In a possible implementation, the initiating device 10 may obtain the distances of other devices connected to the initiating device 10 and the directions of other devices relative to the initiating device 10 through positioning technologies such as Bluetooth, ultra-wideband (UWB), and ultrasound.

[0169] In some examples, the initiating device 10 may display the sub-controls of the drag sensing area in combination with the operation direction of the user's drag operation and the relative positional relationship between the target device 20 and the initiating device 10 .

[0170] For example, the initiating device 10 can simultaneously establish connections with multiple target devices 20, each of which has a different relative positional relationship to the initiating device 10. For example, target device 20A is located to the right of the initiating device 10, target device 20B is located above the initiating device 10, and target device 20C is located to the left of the initiating device 10. In this case, in response to a user dragging the operation object to the lower right of the operation object, the initiating device 10 may display the sub-control 41A corresponding to the target device 20A located to the right, but not display the sub-control 41D.

[0171] In some examples, in response to the user starting a drag operation, the initiating device 10 may display an entry control 40A for indicating a drag perception area. Here, the operation of starting a drag operation may refer to an operation of selecting an operation object and starting to continuously move the operation object.

[0172] In some examples, in response to the user dragging the operating object to the intersection of the drag perception area and the non-drag perception area, the initiating device 10 can display the aforementioned entry control 40A. Referring to Figure 5, the intersection of the drag perception area and the non-drag perception area can be composed of four intersection lines, namely, intersection line 42A, intersection line 42B, intersection line 42C, and intersection line 42D. For example, in response to the user dragging the operating object onto the intersection line 42A, the initiating device 10 can display the sub-control 41A. Alternatively, in response to the user dragging the operating object onto the intersection line 42A, the initiating device 10 can simultaneously display the four sub-controls included in the drag perception area, namely, sub-control 41A, sub-control 41B, sub-control 41C, and sub-control 41D.

[0173] The above technical solution can also be understood as follows: in response to a user dragging an operating object to a position whose distance from the border of the display area of ​​the initiating device 10 is less than or equal to a first distance threshold, the initiating device 10 can display an entry control 40A for indicating a drag-sensing area. For example, in response to a user dragging an operating object to a position whose distance from the right border of the display area is less than or equal to the first distance threshold, the initiating device 10 can display a sub-control 41A. Alternatively, in response to a user dragging an operating object to a position whose distance from the right border of the display area is less than or equal to the first distance threshold, the initiating device 10 can simultaneously display four sub-controls of the entry control 40A.

[0174] When the initiating device 10 is connected to multiple target devices 20 , the multiple target devices 20 may all be located on the same side of the initiating device 10 , or in other words, the multiple target devices 20 may have the same relative positional relationship to the initiating device 10 .

[0175] In some examples, in response to a user dragging an operation object, the initiating device 10 may display identification information of multiple target devices 20 located in the same direction of the initiating device 10 in a sub-area corresponding to the target device 20. In response to a user dragging an operation object 30 to the display area of ​​identification information of one of the target devices 20, the target device 20 may display the operation object 30.

[0176] For example, as shown in FIG6 , target devices 20A, 20B, and 20C are all located above initiating device 10. In response to a user dragging an operation object 30, initiating device 10 may display identification information of target devices 20A, 20B, and 20C in sub-area 41B. In response to the user dragging operation object 30 to the display area for identification information of target device 20B, initiating device 10 transmits information about operation object 30 to target device 20, so that target device 20B can display operation object 30.

[0177] In some examples, in response to the user dragging the operation object 30, the initiating device 10 can display a recommended device in the sub-area corresponding to the target device 20. The recommended device can be determined based on one or more of the distances between the multiple target devices 20 located in the same direction of the initiating device 10 and the initiating device 10, the user's usage habits of the multiple target devices 20, the type of the operation object 30, and the types of the multiple target devices 20.

[0178] For example, taking the distance relationship between the target device 20A, the target device 20B, and the target device 20C and the initiating device 10 in FIG6 as an example, the target device 20A is at the smallest distance from the initiating device 10, and the recommended device may be the target device 20A.

[0179] For example, among the target devices 20A, 20B, and 20C, the user uses the initiating device 10 and the target device 20C simultaneously more often. In this case, the recommended device may be the target device 20C.

[0180] For example, when the operation object 30 is a video file, the display effect of the target device 20A is the best among the target devices 20A, 20B, and 20C. In this case, the recommended device may be the target device 20A.

[0181] In some examples, in response to a user dragging the operation object 30, the initiating device 10 may display identification information of a recommended device in a sub-area corresponding to the target device 20. The recommended device may be determined based on the relative positions of the multiple target devices 20 and the initiating device 10 and the drag direction of the user's drag operation.

[0182] In some examples, the relative positions of the identification information of the plurality of recommended devices are the same as the relative positions of the plurality of recommended devices.

[0183] In some examples, the direction of the line connecting the target device 20 and the initiating device 10 is parallel or approximately parallel to the dragging direction.

[0184] For example, taking the relative positions of the target device 20A, the target device 20B and the target device 20C and the initiating device 10 in Figure 7 as an example, the target device 20A is located at the upper left of the initiating device 10, the target device 20B is located directly above the initiating device 10, and the target device 20C is located at the upper right of the initiating device 10.

[0185] In response to the user dragging the operation object 30 to the upper right of the user interface of the initiating device 10, the initiating device 10 may determine that the target device 20C located at the upper right of the initiating device 10 is a recommended device, and may display identification information of the target device 20C at the position of the sub-area 41B.

[0186] In response to the user dragging the operation object 30 to the upper left of the user interface of the initiating device 10, the initiating device 10 may determine that the target device 20A located at the upper left of the initiating device 10 is a recommended device, and may display identification information of the target device 20A at the position of the sub-area 41B.

[0187] In response to the user dragging the operation object 30 directly above the user interface of the initiating device 10, the initiating device 10 may determine that the target device 20B located directly above the initiating device 10 is a recommended device, and may display identification information of the target device 20B at the position of the sub-area 41B.

[0188] In some examples, in response to the user dragging the operation object 30 , the initiating device 10 may display a recommended device in a sub-area corresponding to the target device 20 . The recommended device may be determined based on the relative positions of the target devices 20 and the initiating device 10 and the postures of the target devices 20 .

[0189] Here, the posture of the target device 20 can also be understood as the placement posture of the target device 20 or the usage posture of the target device 20, such as: horizontal, vertical, oblique, upside down (placed with the screen facing the desktop), etc.

[0190] For example, taking the relative positions and placement postures of the target devices 20A, 20B, and 20C and the starting device 10 in FIG7 as an example, the target devices 20A, 20B, and 20C are all located above the starting device 10, the target device 20A is square, the target device 20B is placed vertically, and the target device 20C is placed diagonally with the screen tilted toward the starting device.

[0191] In a possible implementation, the initiating device may display the placement postures of multiple target devices 20 in the drag sensing area. In other words, the identifiers of the devices displayed in the drag sensing area may also be used to indicate the placement postures of the target devices 20 .

[0192] Because the posture of the target device 20C among the multiple target devices 20 located above makes it easier for the user to view the results of the cross-device drag, in response to the user dragging the operation object 30 toward the top of the user interface of the initiating device 10, the initiating device 10 can determine that the target device 20C in the tilted posture is a recommended device, and can display the identification information of the target device 20C in the sub-area 41B.

[0193] In some examples, in response to a user dragging an object 30, the initiating device 10 may display a recommended device in a subregion corresponding to the target device 20. The recommended device may be determined based on one or more of the relative positional relationship between the multiple target devices 20 and the initiating device 10, the drag direction of the user's drag operation, and the placement of the devices. For example, the recommended device may be determined based on a combination of the relative positional relationship between the multiple target devices 20 and the initiating device 10, the drag direction of the user's drag operation, and the placement of the devices.

[0194] In the case where there is only one recommended device determined by the starting device 10 as described above, in response to the user's operation of dragging the operation object 30, the starting device may not need to display the identification information of the recommended device in the sub-area. The recommending device may directly display the dragged operation object 30 or the user interface containing the operation object 30.

[0195] In one possible implementation, the initiating device 10 can determine the placement postures of multiple target devices 22 based on signals such as magnetic fields emitted by the multiple target devices 22. For example, application number 202310666025.4 provides a method for determining the placement posture of an electronic device. It should be noted that this application does not limit the method for determining the placement posture of the target device 22.

[0196] In the case where the identification information of multiple target devices 20 needs to be displayed at the same time, the display method and display position of the identification information of the multiple target devices 20 can be determined based on one or more of the distances between the multiple target devices 20 located in the same direction of the starting device 10 and the starting device 10, the user's usage habits of the multiple target devices 20, the type of the operation object 30 and the types of the multiple target devices 20, etc.

[0197] FIG9 exemplifies a method for displaying identification information of multiple target devices 20. Unlike the method for displaying identification information of multiple target devices 20 in FIG6 , in which the display areas corresponding to the identification information of different target devices 20 are all equal in size, FIG9 shows that the area of ​​display area 45A is larger than that of display area 45B, and the area of ​​display area 45B is larger than that of display area 45C.

[0198] In some examples, in response to the user's drag operation, the initiating device 10 may display the identification information of the target device 20 with a higher priority at a position in the display area 45A, display the identification information of the target device 20 with a lower priority at a position in the display area 45B, and display the identification information of the target device 20 with the lowest priority at a position in the display area 45C.

[0199] For example, taking the distance relationship between the target device 20A, the target device 20B and the target device 20C and the starting device 10 in Figure 6 as an example, the distance between the target device 20A and the starting device 10 is the smallest, and the distance between the target device 20C and the starting device 10 is the largest. In this case, the starting device 10 can display the identification information of the target device 20A at the position of the display area 45A, display the identification information of the target device 20B at the position of the display area 45B, and display the identification information of the target device 20C at the position of the display area 45C.

[0200] Exemplarily, among the target device 20A, the target device 20B, and the target device 20C, the number of times the user simultaneously uses the starting device 10 and the target device 20C is the greatest, and the number of times the user simultaneously uses the starting device 10 and the target device 20B is the least. In this case, the starting device 10 may display the identification information of the target device 20C at the position of the display area 45A, display the identification information of the target device 20A at the position of the display area 45B, and display the identification information of the target device 20B at the position of the display area 45C.

[0201] For example, if the operation object 30 is a drawing application, both target devices 20A and 20B support touch operations, target device 20C does not support touch operations, and target device 20A also supports stylus operations. The initiating device 10 can determine the display locations for identification information of different target devices 20 based on multiple target settings 20 and the application scenario in which the current user is using the device, or an application scenario in which multiple devices collaborate. For example, the initiating device 10 can display the identification information of target device 20A that supports stylus operations in display area 45A, display the identification information of target device 20B that supports touch operations in display area 45B, and display the identification information of target device 20C that does not support touch operations in display area 45C.

[0202] In some examples, the multiple areas displaying identification information of multiple target devices 20 in the above examples may constitute a subarea of ​​the aforementioned drag sensing area (e.g., the aforementioned first subarea or second subarea, etc.). Alternatively, a subarea of ​​the drag sensing area may include multiple areas for displaying identification information of multiple target devices 20. Alternatively, the interface component for displaying the identification information of multiple target devices 20 and the interface component for displaying the subarea of ​​the drag sensing area may be the same interface component (e.g., subcontrol 41A or subcontrol 41B, etc.). Alternatively, the identification information of multiple target devices 20 may be displayed on a subcontrol of the aforementioned entry control 40A. For example, the identification information of the three target devices in FIG6 may be displayed on subcontrol 41B.

[0203] In some examples, the interface component used to display the identification information of multiple target devices 20 is a device display control that is different from the entry control 40A. The initiating device 10 may display the aforementioned device display control after displaying the entry control 40A. For example, in response to a user dragging the operation object 30, the initiating device 10 may be used to indicate the sub-control 41B of the second sub-area. In response to the user operating the object 30 to remain in the second sub-area for a duration greater than a first time threshold, the initiating device 10 may display the device display control shown in Figure 6, which may be used to display the identification information of the target device 20A, the identification information of the target device 20B, and the identification information of the target device 20C.

[0204] In some examples, the aforementioned operation object 30 may be one or more interface elements or content of interface elements displayed on the user interface of the initiating device 10. For example, the operation object 30 may include one or more of the following: text, images, code, files, interface components, and application windows. Interface components may include buttons, text boxes, sliders, and icons. Application windows may include full-screen windows, split-screen windows, or floating windows.

[0205] For different types of operation objects 30, drag events can be triggered by different operating bodies. Taking the initiating device 10 as a touch screen device as an example, in some examples, a finger and a stylus can be used to trigger drag events for different operation objects 30. Here, the finger and the stylus can be regarded as two different operating bodies.

[0206] Taking the image editing scenario as an example, a long press of a finger or a stylus can trigger a drag event of the image to be edited. Since the stylus can be equipped with a pressure sensor, a hard press of the stylus can be used to trigger a drag event of image elements such as color and shape in the image to be edited.

[0207] In some examples, finger operations may include single-finger operations, multi-finger operations, palm operations, or knuckle operations, etc. In the single-finger operations, operations with different fingers may be used to trigger drag events for different operation objects 30 , and multi-finger operations may include two-finger operations, three-finger operations, or four-finger operations, etc. Multi-finger operations with different numbers of fingers may also be used to trigger drag events for different operation objects 30 .

[0208] Taking the text editing scenario as an example, a single-finger operation can be used to trigger a drag event of the text in the file to be edited, a three-finger operation can be used to trigger a drag event of taking a screenshot of the text editing interface, and a palm operation can be used to trigger a drag event of the text editing window.

[0209] In some examples, for the same operating body, different operation modes can also trigger drag events of different operating objects 30. For example, a single-finger hard press can be used to trigger a drag event of a recognizable object in an image, and a single-finger long press can be used to trigger a drag event of an image.

[0210] Taking the image editing scenario in FIG. 10 as an example, in the left schematic diagram, in response to a user single-finger tapping the image to be edited, initiating device 10 may determine that operation object 30 is the image to be edited, hereinafter referred to as operation object 31A for ease of distinction. In the right schematic diagram, in response to a user single-finger re-pressing the image to be edited, initiating device 10 may determine that the operation object is a recognizable object in the image to be edited, hereinafter referred to as operation object 31B for ease of distinction.

[0211] In response to a drag operation on a different type of operation object 30, the initiating device 10 may display, at the location of the drag sensing area, a processing method corresponding to the type of operation object 30 or an application method of the target device 20 corresponding to the type of operation object 30. For example, the application method of the target device 20 may include mirroring, extending, and sharing.

[0212] In some scenarios, the application mode of a device can also be understood as the display mode of the operation object.

[0213] 11 , in response to a user dragging an application window in the user interface of the initiating device 10 , the initiating device 10 may display three application modes of the target device 20 at the position of the sub-area 41B, namely, mode 1A, mode 1B, and mode 1C.

[0214] Among them, method 1A can refer to using the target device 20 as a mirror display device, that is, the content displayed on the starting device 10 and the target device 20 is the same. In this case, in response to the user's drag operation of dragging the application window to the display area of ​​method 1A, the starting device 10 and the target device 20 can simultaneously display the application window dragged by the user, and the status of the application windows displayed by the two devices is the same.

[0215] Mode 1B may refer to using the target device 20 as an extended display device, that is, the display screen of the starting device 10 and the display screen of the target device 20 together constitute the display area of ​​the starting device 10. In this case, in response to the user dragging the application window to the display area of ​​mode 1B, the target device 20 may display the application window dragged by the user, and the starting device 10 may not display the application window dragged by the user.

[0216] Method 1C may refer to using the target device 20 as a data sharing device, and the data of the initiating device 10 may be transmitted to the target device 20. In this case, in response to the user's dragging operation of dragging the application window to the display area of ​​method 1C, the initiating device 10 may send information related to the application window to the target device 20, and the target device 20 may display a file indicating the information of the aforementioned application window, etc.

[0217] In one possible implementation, before the user performs the drag operation, the application mode of the target device 20 has already been set, or in other words, a default value is set for the application mode of the target device 20. In response to the user's drag operation, the initiating device 10 may perform the drag operation on the operation object 30 according to the set application mode of the target device 20. For example, the application mode of the target device 20 may be set to the aforementioned mode 1B or the "extended" application mode. In response to the user's operation of dragging the operation object 30, the target device 20 may display the operation object 30, while the initiating device 10 may not display the operation object 30.

[0218] In one possible implementation, before the user performs the drag operation, the application mode of the target device 20 has already been set. In response to the user's dragging and reselecting the application mode of the target device 20, the initiating device 10 may perform the drag operation on the operation object 30 according to the user's reselected application mode of the target device 20. For example, in response to the user dragging the operation object 30 into the drag sensing area and leaving it there for longer than a second time threshold, the initiating device 10 may display one or more application modes of the target device 20 at the location of the drag sensing area. In response to the user dragging the operation object into the display area for the one or more application modes, the initiating device 10 may determine the application mode of the target device 20 reselected by the user and perform the drag operation on the operation object 30 according to the user's reselected application mode.

[0219] In some examples, in response to a user dragging an operation object to the display area of ​​the aforementioned method 1A, method 1B, or method 1C, the initiating device 10 can send indication information to the target device 20, which is used to indicate the display method of the operation object selected by the user, or the application method of the target device 20.

[0220] During each cross-device drag operation, the user can reselect the application mode of the target device 20, which is beneficial to improve the efficiency of the collaborative work between the starting device 10 and the target device 20 to a certain extent, and enhance the user's experience of cross-device drag operations.

[0221] As described above, when the operation object 30 is moved to the drag sensing area of ​​the initiating device 10 , the target device 20 may display the operation object 30 .

[0222] In some examples, before the operation object 30 is moved to the drag sensing area of ​​the initiating device 10, in response to the user stopping dragging or releasing the operation object 30, the operation object 30 may retract to its initial position. For example, referring to FIG4 , in response to the user releasing the operation object 30 before moving the operation object 30 to the entry area, the initiating device 10 may display the operation object 30 at the initial position of the operation object 30 indicated by the dashed line, while the target device 20 may not display the operation object 30, or in other words, the target device 20 may not perform an operation associated with the operation object 30.

[0223] Continuing with FIG4 , the user interface of the target device 20 may include an exit area, or in other words, the target device 20 may display an exit control 40B, which may be used to indicate the exit area on the target device 20. In some scenarios, the exit area may also be referred to as a response area. Similar to the drag sensing area of ​​the initiating device 10 described above, the exit area may be located at the edge of the display area of ​​the target device 20. In other words, the exit control 40B may be located on the periphery of the display area of ​​the target device 20.

[0224] In some examples, the exit area of ​​the target device 20 may include multiple sub-areas (e.g., four) similar to the drag sensing area of ​​the initiating device 10 in FIG. 5 , and these multiple sub-areas may be displayed on the four edges of the display area of ​​the target device 20. In other words, the exit control 40B may include multiple sub-controls, each of which may be used to indicate the multiple sub-areas of the exit area, and the multiple sub-controls may be displayed on all four sides of the display area of ​​the target device 20.

[0225] In some examples, the exit control 40B of the target device 20 may be displayed only on the edge of the display area of ​​the target device 20 close to the initiating device 10. Taking the relative position relationship between the initiating device 10 and the target device 20 in FIG4 as an example, the target device 20 is located to the right of the initiating device 10, and the exit control 40B of the target device 20 may be displayed on the left edge of the display area.

[0226] In some examples, the target device 20 may display the operation object 30 at the location of the aforementioned exit area.

[0227] For example, the target device 20 may highlight the operation object 30 at the exit area to remind the user. For example, the target device 20 may highlight the operation object 30 in a bright color, display the operation object 30 with animation effects (such as shaking, deformation, etc.), enlarge the operation object 30, or hover the operation object 30 at the exit area.

[0228] Regarding the case where the target device 20 displays the operation object 30 at the exit area, in some examples, as shown in FIG12 , in response to the user continuing to drag the operation object 30 on the target device 20, the target device 20 may display the operation object at the user-selected target location 30D. In FIG12 , the target object 30 is dragged from an initial location 30A on the initiating device 10 to an entry location 30B of the exit area, thereby triggering a cross-device drag and displaying the operation object 30 at an exit location 30C of the exit area of ​​the target device 20. In response to the user continuing to drag the operation object 30 on the target device 20, the target device 20 may display the operation object 30 at the target location 30D.

[0229] In some examples, the entry position 30B may be the position where the user releases the operation object 30 after stopping the drag operation.

[0230] In some examples, in response to the user's operation of continuously dragging the operation object 30 from the initial position 30A to the drag sensing area until reaching the edge of the display area of ​​the initiating device 10, the initiating device 10 can determine the position information where the dragged object 30 leaves the display area, for example, the coordinate information of the position where the dragged object 30 leaves the display area. In this case, the position where the dragged object 30 leaves the display area can be regarded as the aforementioned entry position 30B.

[0231] The location of the operation object 30 displayed on the target device 20, exit location 30C, can be determined based on information such as entry location 30B. For example, the distances between exit location 30C and the top and bottom edges of the target device 20 are L1 and L2, respectively, and the distances between the entry location and the top and bottom edges of the originating device 10 are L3 and L4, respectively. Thus, L1:L2 = L3:L4.

[0232] For example, let the width of the display area of ​​the initiating device 10 be w1, and the width of the display area of ​​the target device 20 be w2. With the length of the devices as the x-axis, the width as the y-axis, and the lower left corner of the display area as the coordinate origin, the coordinates of the entry position 30B of the operation object 30 on the initiating device 10 are (x1, y1), and the coordinates of the exit position 30C of the operation object 30 on the target device 20 are (x2, y2). For example, if the target device 20 is to the right of the initiating device 10 and the user drags the operation object 30 from the initial position 30A to the lower right of the initiating device 10, then y2 = y1 * w2 / w1, and x2 = 0.

[0233] In some examples, the initiating device 10 may send indication information to the target device 20 , where the indication information may be used to indicate an entry position, which may be a position where the operating object 30 is released in the triggering area.

[0234] In some examples, in response to the user dragging the operation object 30 to the target application on the target device 20 , the target device 20 may display a user interface of the target application, which may be used to open or process the operation object 30 .

[0235] Here, the target application in the target application to which the operation object 30 is dragged can be understood as the user interface of the target application, the icon of the target application, or other interface elements capable of triggering the calling of the target application.

[0236] Exemplarily, as shown in Figure 13, the operation object 30 can be a picture file. In response to the user dragging the picture file to the application icon of the "Gallery" application 50, the target device 20 can display the user interface 51 of the "Gallery" application 50 shown below in Figure 13, which can be used to display the aforementioned picture file.

[0237] Exemplarily, the operation object 30 may be text information. In response to the user dragging the text information to the user interface of the "text editing" application, the target device 20 may display another user interface of the "text editing" application, on which the aforementioned text information dragged by the user may be displayed.

[0238] The operation object 30 located at the initial position 30A can also be called the operation object 30 in the starting state, the operation object 30 located at the entry position 30B can also be called the operation object 30 in the entry state, the operation object 30 located at the exit position 30B can also be called the operation object 30 in the exit state, and the operation object 30 located at the target position 30D can also be called the operation object 30 in the target state.

[0239] In the case where the target device 20 displays the operation object 30 at the exit area, in some examples, as shown in FIG14 , in response to the operation object 30 not being operated for a preset period of time, the target device 20 may no longer display the operation object 30. In other words, the target device 20 may be restored to the state before the cross-device drag operation was performed. The initiating device 10 may display the operation object 30 at the initial position 30A, or in other words, the initiating device 10 may display the operation object 30 in the initial state. In other words, the initiating device 10 may be restored to the state before the cross-device drag operation was performed.

[0240] In some examples, the operation object 30 may be displayed on a default interface of the target device 20. Alternatively, the operation object 30 may be stored at a default address. The default interface or default address may be set by the user or preset by the target device 20. Furthermore, the default interface or default address may be set separately for different types of operation objects 30.

[0241] 15 , the operation object 30 may be an application window, and the default interface may be the interface currently displayed on the target device 20. In other words, in response to the user dragging the application window from the initiating device 10 to the target device 20, the target device 20 displays the user interface of the application window on top of the current user interface.

[0242] In the case where the operation object 30 is an application window, the target device 20 may display the application window in different preset ways according to the type of the target device 20 .

[0243] For example, the target device 20 may display the user interface corresponding to the application window in different ways based on the size of its own display screen. If the screen size of the target device 20 is greater than the first area threshold, the target device 20 may display the user interface of the operation object 30 in the form of a floating window. If the screen size of the target device 20 is less than or equal to the first area threshold, the target device 20 may display the user interface of the operation object 30 in the form of a full-screen window. For example, the aforementioned first area threshold may be represented by the diagonal length of the screen, for example, the first area threshold is 8 inches.

[0244] For example, in FIG. 15 , the diagonal length of the screen of the target device 20 is 11 inches. In this case, the target device 20 may display the user interface of the operation object 30 in the form of a floating window.

[0245] For example, in FIG. 16 , the diagonal length of the screen of the target device 20 is 6.7 inches. In this case, the target device 20 may display the user interface of the operation object 30 in the form of a full-screen window.

[0246] In the case where the operation object 30 is an application window, the target device 20 may determine whether to display different windows of the application corresponding to the operation object 30 in a split-window display manner according to the type of the target device 20 .

[0247] Exemplarily, the application window corresponding to the operation object 30 may include a first window and a second window. When the screen size of the display screen of the target device 20 is greater than or equal to the second threshold, in response to the user's operation of dragging the operation object across devices, the target device 20 may display the first window and the second window. When the screen size of the display screen of the target device 20 is less than the second threshold, in response to the user's operation of dragging the operation object across devices, the target device 20 may only display the first window or the second window. In some examples, when the operation object 30 is a desktop plug-in, a quick application, or a desktop component, etc., in response to the user performing an operation of dragging across devices, the target device 20 may also display a window corresponding to the operation object 30 in the current user interface.

[0248] For example, the operation object 30 may be a target file, and the default interface may be an interface of an application that can open the target file. In other words, in response to a user dragging the target file from the initiating device 10 to the target device 20, the target device 20 may display a user interface of the target application, which can be used to open or process the target file.

[0249] For example, the operation object 30 may be an image file, and the default interface may be the user interface 51 of the "Gallery" application 50 in FIG13 , which may be used to display the image file. In other words, in response to the user dragging the image file from the initiating device 10 to the target device 20, the target device 20 may display a user interface for displaying the image file.

[0250] For another example, the operation object 30 may be a text message, and the default interface may be the user interface of a "text editing" application, which may display the aforementioned text message dragged by the user. In other words, in response to the user dragging the text message from the initiating device 10 to the target device 20 across devices, the target device 20 may display the user interface of an application for editing the text message.

[0251] In some examples, the target device 20 at the exit area 40B may display one or more processing methods corresponding to the operation object 30 or application methods of the target device 20 corresponding to the type of operation object 30. For example, the application methods of the target device 20 may include mirroring, extension, and sharing.

[0252] For example, as shown in FIG17 , in response to a user dragging an application window in the user interface of the initiating device 10 to the drag sensing area, the target device 20 may display an operation object 30. For example, the target device 20 may display the operation object 30 in the center of the user interface. In response to the user selecting the operation object 30, the target device 20 may display three application modes of the target device 20 at the exit area 40B: Mode 1A, Mode 1B, and Mode 1C.

[0253] Among them, mode 1A may refer to using the target device 20 as a mirrored display device, that is, the content displayed on the initiating device 10 and the target device 20 is the same. In this case, in response to a user dragging an application window to the display area of ​​mode 1A, the initiating device 10 and the target device 20 can simultaneously display the user-dragged application window, and the application windows displayed on both devices are in the same state. Mode 1B may refer to using the target device 20 as an extended display device, that is, the display screen of the initiating device 10 and the display screen of the target device 20 together constitute the display area of ​​the initiating device 10. In this case, in response to a user dragging an application window to the display area of ​​mode 1B, the target device 20 can display the user-dragged application window, and the initiating device 10 may not display the user-dragged application window. Method 1C may refer to using the target device 20 as a data sharing device, and the data of the initiating device 10 may be transmitted to the target device 20. In this case, in response to the user's dragging operation of dragging the application window to the display area of ​​method 1C, the initiating device 10 may send information related to the application window to the target device 20, and the target device 20 may display a file indicating the information of the aforementioned application window, etc.

[0254] For different types of operation objects 30 , or for different types of cross-device drag tasks, or for different application modes of the target device 20 , the target device 20 may display the user interface of the operation object 30 in different ways.

[0255] In some examples, the operation object 30 may be a window of an application that needs to be displayed in real time. In response to the user's cross-device drag operation, the target device 20 may display the window of the application. Figure 16 above is an example. In some examples, the operation object 30 may be a data file such as an image file or a video file. The target device 20 may not display the transfer process of these data files in real time. In this case, in response to the user's cross-device drag operation, the target device 20 may continue to display the currently displayed user interface, or in other words, the target device 20 may not display the transfer process of the operation object 30 in real time.

[0256] In some examples, a screen sharing task is performed between the initiating device 10 and the target device 20, and a user's cross-device dragging operation on the operation object 30 can trigger the screen sharing task. In this case, in response to the user's cross-device dragging operation, the target device 20 can display the content displayed on the screen of the initiating device 10, or in other words, the target device 20 can display the transfer process of the cross-device dragged object in real time. In some examples, a data sharing task is performed between the initiating device 10 and the target device 20, and a user's cross-device dragging operation on the operation object 30 can trigger the data sharing task. In this case, in response to the user's cross-device dragging operation, the target device 20 may not display the transfer process of the operation object 30 in real time.

[0257] In some examples, in response to the user selecting "Method 1A" or "Method 1B" as the application mode of the target device 20 during the cross-device drag operation, or in other words, in response to the user selecting to apply the target device 20 in a mirrored or extended manner, the target device 20 can display the transmission process of the cross-device drag operation object 30 in real time; in some examples, in response to the user selecting "Method 1C" as the application mode of the target device 20 during the cross-device drag operation, or in other words, in response to the user selecting to apply the target device 20 in a shared manner, the target device 20 may not display the transmission process of the cross-device drag operation object 30 in real time.

[0258] In some scenarios, the above technical solution can also be understood as follows: if the operation object 30 involves (or includes) a synchronous transmission task, the target device 20 can explicitly display the transmission result of the operation object 30; if the operation object 30 does not involve (or does not include) a synchronous transmission task, the target device 20 may not explicitly display the transmission result of the operation object 30. In other words, if the operation object 30 only involves (or only includes) an asynchronous transmission task, the target device 20 may not explicitly display the transmission result of the operation object 30.

[0259] Taking the aforementioned "Method 1A" of the target device 20 as an example, the display screen of the target device 20 can display the content displayed on the display screen of the initiating device 10. In other words, the initiating device 10 can project the content onto the target device 20. In this case, the target device 20 can be considered to be explicitly displaying the transmission result of the operation object 30.

[0260] Referring to Figure 18 , operation object 30 is a text message. In response to a user dragging operation object 30 to the "Memo" application desktop widget on the user interface of target device 20, target device 20 may create a new memo containing the text message in the desktop widget. In this case, target device 20 may be considered to be non-explicitly displaying the transmission result of operation object 30.

[0261] Displaying the operation objects of the cross-device drag operation in different ways is conducive to more reasonable scheduling of software and hardware resources of the target device 20, and is conducive to improving the efficiency of the collaborative work between the initiating device 10 and the target device 20.

[0262] In the above embodiment, in response to a user dragging an operation object 30 on the initiating device 10 to the entry area and stopping dragging or releasing the operation object 30, the initiating device 10 can trigger a task to perform the cross-device drag operation, thereby achieving collaboration with the target device 20. The implementation of this technical solution can, to a certain extent, reduce the adverse effects of a continuous drag stroke on the cross-device drag operation, thereby improving the efficiency of collaboration between the initiating device 10 and the target device 20.

[0263] In some scenarios, the initiating device 10 and the target device 20 may be placed side by side. In this case, the success rate of the cross-device drag operation may be improved by determining the user's intention to continue dragging on the initiating device 10. In other words, in this case, the success rate of the cross-device drag operation may be improved by determining whether the user needs to perform the cross-device drag operation.

[0264] Figure 19 shows a method for cross-device collaboration provided by an embodiment of the present application. When an operation object is dragged to the trigger area of ​​the first device, the second device connected to the first device can display the operation object. In this method, the user does not need to continuously drag the operation object from the first device to the second device to trigger the cross-device dragging of the operation object. This method is conducive to improving the efficiency of collaborative work between the first device and the second device.

[0265] S101 : In response to a drag operation on an operation object on a first device, move the operation object.

[0266] The operation object may be one or more interface elements or the content of interface elements on the starting interface of the user interface of the first device. For example, the operation object may include one or more of the following: text, images, code, files, interface components, and application windows. Interface components may include buttons, text boxes, sliders, and icons. Application windows may include full-screen windows, split-screen windows, or floating windows.

[0267] The object used to drag an object can be called an operator. The operator can be a mouse, trackpad, digitizer, finger, or stylus, and different quantities of the same operator can be considered different operators. In some examples, different fingers or different numbers of fingers can be considered different operators. Different operators can be used to trigger the dragging of different objects.

[0268] For example, in a picture editing interface, a finger can trigger a drag operation of a picture to be edited, and a stylus can trigger a drag operation of picture elements such as colors and lines in the picture to be edited.

[0269] Different operation modes can be used to trigger the drag operation of different types of operation objects, and the operation modes here may include: tap, long press, hard press or double click, etc.

[0270] For example, in the picture editing interface, a finger tap can trigger the drag operation of the picture editing window, a long press of the finger can trigger the drag operation of the picture to be edited, and a hard press of the finger can trigger the drag operation of a recognizable object in the picture to be edited.

[0271] In some examples, the direction in which the user moves the operation object can be the operation direction. In one possible implementation, the first device can determine the operation direction of the user's drag operation based on capacitance information of its own display screen. The first device can determine the second device based on the operation direction.

[0272] S102: Determine whether the operation object is in a triggering area on the first device.

[0273] The trigger area may refer to an area on the first device that can trigger the operation object to perform cross-device dragging. The trigger area may be the drag sensing area in the aforementioned embodiment.

[0274] In some examples, the trigger area may be located at the periphery of the display area of ​​the first device. Exemplarily, the aforementioned drag sensing area may include a first sub-area, a second sub-area, a third sub-area, and a fourth sub-area.

[0275] The first device may indicate the aforementioned trigger area through an indicating control, and the indicating control may include multiple sub-controls, and the multiple sub-controls may be used to indicate different positions of the trigger area respectively.

[0276] Exemplarily, the indication control may be the aforementioned entry control 40A, which may be used to indicate the entry area. The entry control 40A may include sub-controls 41A, 41B, 40C, and 40D. Sub-control 41A may be used to indicate the first sub-area, sub-control 41B may be used to indicate the second sub-area, sub-control 41C may be used to indicate the third sub-area, and sub-control 41D may be used to indicate the fourth sub-area.

[0277] In some examples, in response to a user dragging an operation object, the first device may display an indicator control around the display area, which may be used to indicate that the trigger area is located around the display area of ​​the first device.

[0278] In some examples, in response to a user dragging an operation object, the first device may display an indicator control on the edge of the display area close to the second device, and the indicator control may be used to indicate that the trigger area is located at the edge of the display area of ​​the first device close to the second device.

[0279] In some examples, in response to a user dragging an operation object in a first direction, the first device may display an indicator control at an edge of the display area pointing upward in the first direction. For example, the first direction may be rightward, in which case the first device may display the indicator control at the right edge of the display area. For example, the first direction may be upward-leftward, in which case the first device may display the indicator control at both the upper edge and the left edge of the display area.

[0280] The distance between the boundary of the trigger area, the area outside the trigger area, and the outer boundary of the display area of ​​the first device can be referred to as a first distance threshold, or in other words, the width of the trigger area can be the first distance threshold. In some examples, in response to a user dragging an operation object to a point where the distance from the outer boundary of the display area of ​​the first device is less than or equal to the first distance threshold, the first device can display the aforementioned indicator control to indicate the trigger area.

[0281] In some examples, the location of the trigger area may also display indication information for indicating one or more application modes of the second device. For example, the application modes of the second device may include mirroring, extension, and sharing.

[0282] For example, in response to a user dragging an operation object, the first device may display one or more application modes of the second device at the location of the trigger area. In response to a user dragging the operation object to the display area of ​​one of the application modes, the second device displays the operation object according to the application mode selected by the user.

[0283] In some examples, the second device may be based on the relative position relationship between the second device and the first device, the placement posture of the second device, or the direction in which the operation object is dragged.

[0284] In some examples, the first device can establish connections with multiple electronic devices, and the location of the trigger area can also display the identification information of multiple second devices connected to the first device. The multiple second devices can be electronic devices with the same location relationship as the first device. The second device can be determined based on one or more of the distance between the multiple second devices and the first device, the user's usage habits of the multiple second devices, the type of the operation object, and the type of the multiple second devices. For details, please refer to the relevant description above and will not be repeated here.

[0285] In one possible implementation, the above-mentioned indication control for indicating the trigger area, the display control for displaying indication information indicating the application mode of the second device, and the display control for displaying identification information of multiple second devices can be the same control, or these controls can be multiple independent controls, which is not limited in this application.

[0286] S103: Display the operation object on the second device.

[0287] The second device may refer to an electronic device that establishes a connection with the first device, and the connection method may be a wired connection or a wireless connection, wherein the wireless connection protocol may include one or more of the following: Wireless Fidelity protocol, Bluetooth protocol, ZigBee protocol, short-range wireless communication protocol or various cellular network protocols, etc.

[0288] In some examples, in response to a user dragging an operation object to a trigger area of ​​the first device, the operation object is displayed on the user interface of the second device.

[0289] The second device may be the recommended device determined in the aforementioned embodiment based on the distance from the first device, the user's usage habits of multiple second devices, the type of the operation object, and the types of the multiple second devices. Alternatively, the second device may be selected by the user based on identification information of multiple second devices displayed on the first device.

[0290] In other words, in response to an operation object being dragged to the display area of ​​the identification information of the second device on the user interface of the first device, the operation object is displayed on the user interface of the second device. Alternatively, in response to an operation object being dragged to a trigger area on the first device near the side of the second device, the operation object is displayed on the user interface of the second device.

[0291] In some examples, the second device may display the operation object in the response area. Specifically, the second device may display the operation object at the exit position of the response area. The exit position may be determined based on the entry position of the operation object on the first device. The entry position may be the position of the operation object when the cross-device drag is triggered on the first device, or the entry position may be the position where the operation object leaves the display area of ​​the first device. For the method of determining the exit position based on the entry position, please refer to the relevant description in the previous text and will not be repeated here.

[0292] In some examples, the second device may be displayed at a default location, which may include a default interface and / or a default address. When the operation object is a window, the default interface may be the current user interface of the second device. When the operation object is an image, text message, video file, or audio file, the default interface may be the user interface of an application for editing, viewing, or processing these files.

[0293] In some examples, the second device can display the application window in a preset manner, which is determined according to the type of the second device. The preset manner can be any one of a floating window, a split-screen window, a multi-window, or a full-screen window.

[0294] In some examples, the second device may determine how to display the operation object based on the size of its own display screen. When the size of the second device's display screen is greater than a first area threshold, the second device may display the user interface containing the operation object in the form of a floating window. When the size of the second device's display screen is less than or equal to the first area threshold, the second device may display the user interface containing the operation object in the form of a full-screen window. The operation object may be an application window, an image, or text information, etc.

[0295] In some examples, the response area on the user interface of the second device may also display indicative information of the application mode of the second device. For example, the application mode of the second device may include mirroring, extension, and sharing.

[0296] Exemplarily, in response to a user selecting an operation object on the user interface of the second device, the second device may display indication information of one or more application modes of the second device at a location in the response area. In response to a user dragging an operation object on the user interface of the second device to a display area of ​​one of the application modes, the second device displays the operation object according to the application mode selected by the user.

[0297] In some examples, for different types of operation objects, the second device can display the transmission results of the operation objects in different display modes. For example, if the operation object involves a synchronous transmission task, the second device can explicitly display the transmission results of the operation object. If the operation object does not involve a synchronous transmission task, the second device can non-explicitly display the transmission results of the operation object.

[0298] In some examples, in response to the operation object not being operated within a preset time period, the second device may no longer or stop displaying the operation object. In other words, in response to the user not operating the operation object within a preset time period, the second device may be restored to a state before the cross-device drag operation was performed.

[0299] S104: Display the operation object at the initial position of the first device.

[0300] In some examples, in response to the user stopping dragging or releasing the operation object before dragging the operation object to the trigger area, the first device displays the operation object at the initial position of the operation object.

[0301] In some examples, when a user is dragging an operation object, in a scenario where the user is required to select a second device from multiple devices, in response to the user not selecting the second device within a preset time period, the first device displays the operation object at the initial position of the operation object.

[0302] In some examples, while a user is dragging an operation object, in a scenario where the user is required to select one of multiple application modes as the application mode of the second device, in response to the user not selecting the second device within a preset time period, the first device displays the operation object at its initial position. Here, identification information for the multiple application modes can be displayed in a trigger area of ​​the first device, or alternatively, identification information for the multiple application modes can be displayed in a response area of ​​the second device.

[0303] In some examples, when the user interface of the second device already displays an operation object, in response to the user not operating the operation object within a preset time period, the second device can be restored to the state before performing the cross-device drag operation, and the first device displays the operation object at its initial position.

[0304] Figure 20 shows a cross-device collaboration apparatus 2000 provided in an embodiment of the present application. The apparatus 2000 may have the functions of the electronic device in the above method embodiment and may be used to execute the steps performed by the functions of the electronic device in the above method embodiment. The functions may be implemented in hardware, or in software or hardware executing corresponding software implementations. The hardware or software may include one or more modules corresponding to the above functions.

[0305] In a possible implementation, the cross-device collaboration apparatus 2000 may include an acquisition module 2010 and a processing module 2020 , and the acquisition module 2010 and the processing module 2020 are coupled to each other.

[0306] In some examples, the acquisition module 2010 can be used to support the first device or the initiating device in the aforementioned embodiments in acquiring user input, such as acquiring the user's drag operation on the operation object.

[0307] The processing module 2020 is used to support the first device or the initiating device to execute the processing actions in the above method embodiment, such as displaying an entry control, etc.

[0308] Optionally, the cross-device collaboration apparatus 2000 may further include a storage unit 2030 for storing program codes and data of the cross-device collaboration apparatus 2000 .

[0309] In some examples, the acquisition module 2010 can be used to support the second device or target device in the aforementioned embodiments to obtain user input, such as obtaining the user's operation of selecting an application mode of the second device on the second device.

[0310] The processing module 2020 is used to support the electronic device in executing the processing actions in the above method embodiments, such as displaying an exit control.

[0311] Optionally, the cross-device collaboration apparatus 2000 may further include a storage unit 2030 for storing program codes and data of the cross-device collaboration apparatus 2000 .

[0312] FIG21 illustrates an electronic device 2100 provided in an embodiment of the present application. As shown in the figure, the electronic device 2100 includes at least one processor 2110 and a transceiver 2120. The processor 2110 is coupled to a memory and configured to execute instructions stored in the memory to control the transceiver 2120 to transmit and / or receive signals.

[0313] Optionally, the electronic device 2100 further includes a memory 2130 for storing instructions.

[0314] In some embodiments, the processor 2110 and the memory 2130 may be combined into a processing device, and the processor 2110 is configured to execute program codes stored in the memory 2130 to implement the above functions. In specific implementations, the memory 2130 may also be integrated into the processor 2110 or independent of the processor 2110.

[0315] In some embodiments, the transceiver 2120 may include a receiver (or receiver) and a transmitter (or transmitter).

[0316] The transceiver 2120 may further include an antenna, and the number of antennas may be one or more. The transceiver 2120 may be a communication interface or an interface circuit.

[0317] When the electronic device 2100 is a chip, the chip includes a transceiver module and a processing module. The transceiver module may be an input / output circuit or a communication interface; the processing module may be a processor, microprocessor, or integrated circuit integrated on the chip.

[0318] This embodiment also provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the cross-device collaboration method in the above-mentioned embodiment.

[0319] This embodiment also provides a computer program product. When the computer program product is run on a computer, it enables the computer to execute the above-mentioned related steps to implement the cross-device collaboration method in the above-mentioned embodiment.

[0320] In addition, embodiments of the present application further provide a device, which may be a chip, component, or module, and may include a processor and a memory connected thereto. The memory is configured to store computer-executable instructions, and when the device is running, the processor may execute the computer-executable instructions stored in the memory to cause the chip to perform the cross-device collaboration method described in each of the above method embodiments.

[0321] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0322] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0323] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0324] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0325] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0326] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0327] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A cross-device collaboration method, applied to a first device, characterized in that: include: Displaying a starting interface, wherein the starting interface includes an operation object; In response to an operation of dragging the operation object to a trigger area and releasing it, sending the operation object to the second device, wherein the trigger area is located outside the display area of the first device; The first device and the second device are connected to each other, and the second device is determined according to one or more of the following: the relative position relationship between the second device and the first device, the placement posture of the second device, or the direction in which the operation object is dragged.

2. The method according to claim 1, characterized in that Before sending the operation object to the second device in response to the operation of dragging the operation object to the trigger area and releasing it, the method further includes: Displaying one or more shortcuts in the trigger area, wherein the shortcuts are used to indicate a display mode of the operation object; In response to the operation of dragging the operation object to the display area of the first shortcut, indication information is sent to the second device, the indication information is used to indicate the first display mode, the first shortcut is used to indicate the first display mode, and the one or more shortcuts include the first shortcut.

3. The method according to claim 1 or 2, characterized in that The sending of the operation object to the second device in response to an operation of dragging the operation object to the trigger area and releasing the operation object includes: displaying identification information of the second device in the trigger area; In response to an operation of dragging the operation object to a display area of identification information of the second device and releasing the operation object, the operation object is sent to the second device.

4. The method according to any one of claims 1 to 3, characterized in that In response to an operation of dragging the operation object to a trigger area and releasing it, before sending the operation object to the second device, the method further includes: In response to an operation performed on a first object using a first operation mode, selecting a portion of the first object as the operation object; In response to an operation performed on the first object using a second operation mode, selecting the first object as the operation object; The first operation mode is different from the second operation mode.

5. The method according to any one of claims 1 to 4, characterized in that Before sending the operation object to the second device in response to the operation of dragging the operation object to the trigger area and releasing it, the method further includes: In response to a distance between the operation object and an edge of the display area of the first device being less than a first threshold, an indication control is displayed, where the indication control is used to indicate the trigger area.

6. The method according to any one of claims 1 to 5, characterized in that The trigger area is located at an edge of a display area of the first device on a side close to the second device.

7. A cross-device collaboration method, applied to a first device, characterized in that: include: Displaying a starting interface, wherein the starting interface includes an operation object; In response to a first drag operation on the operation object along a first drag direction, displaying a first trigger area on a periphery of the starting interface corresponding to the first drag direction; In response to a release operation on the operation object in the first triggering area, the operation object is sent to a second device connected to the first device.

8. The method according to claim 7, characterized in that The method further comprises: In response to a second drag operation on the operation object along a second drag direction, displaying a second trigger area on a periphery of the starting interface corresponding to the second drag direction; In response to a release operation on the operation object in the second triggering area, the operation object is sent to a third device connected to the first device.

9. The method according to claim 7, characterized in that The method further comprises: In response to an operation of dragging the operation object to the first trigger area, displaying one or more device identifiers in the first trigger area, the one or more device identifiers being used to indicate candidate devices, the candidate devices being located on a side of the first device close to the first trigger area, the candidate devices including the second device; The sending, in response to a release operation on the operation object in the first triggering area, the operation object to a second device connected to the first device, includes: In response to an operation of dragging the operation object to a display area of a device identifier of the second device and releasing the operation object, the operation object is sent to the second device.

10. The method according to claim 9, characterized in that There are multiple device identifiers, and the relative positions of the multiple device identifiers are the same as the relative positions of the multiple candidate devices corresponding to the multiple device identifiers.

11. The method according to claim 9 or 10, characterized in that The device identifier is also used to indicate the placement posture of the candidate device.

12. A cross-device collaboration method, characterized in that: include: The first device displays a starting interface, where the starting interface includes an operation object; In response to an operation of dragging the operation object to a trigger area of the first device and releasing the operation object, the second device displays the operation object, the trigger area being located outside the display area of the first device, and the second device determining the operation object based on one or more of the following: a relative positional relationship between the second device and the first device, a placement posture of the second device, or a direction in which the operation object is dragged; The first device and the second device are connected to each other.

13. The method according to claim 12, characterized in that In response to an operation of dragging the operation object to a trigger area of the first device and releasing the operation object, the second device displaying the operation object includes: The second device displays the operation object in a response area, and the response area is located outside a display area of the second device.

14. The method according to claim 13, characterized in that The second device displays the operation object in a response area, including: The second device displays the operation object at an exit position of the response area, where the exit position is determined according to an entry position, and the entry position is a position where the operation object is released from the triggering area.

15. The method according to claim 13 or 14, characterized in that The response area further displays one or more shortcuts, where the shortcuts are used to indicate a display mode of the operation object. The method further includes: In response to the operation of dragging the operation object to the display area of the second shortcut, the second device displays the operation object in a second display mode, the second shortcut is used to indicate the second display mode, and the one or more shortcuts include the second shortcut.

16. The method according to any one of claims 12 to 15, characterized in that The method further comprises: In a case where the second device displays the operation object, in response to the operation object not being operated within a preset time period, the second device stops displaying the operation object.

17. The method according to any one of claims 12 to 16, characterized in that The operation object is an application window, and in response to an operation of dragging the operation object to a trigger area of the first device and releasing the operation object, the second device displays the operation object, including: The second device displays the application window in a preset manner, where the preset manner is determined according to the type of the second device, and the preset manner includes any one of a floating window, a split-screen window, a multi-window, or a full-screen window.

18. The method according to any one of claims 12 to 17, characterized in that In response to an operation of dragging the operation object to a trigger area of the first device and releasing the operation object, the second device displays the operation object, including: The second device displays a user interface of a target application, where the target application is used to edit the operation object.

19. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory is used to store program instructions, and the processor is used to call the program instructions to execute the method described in any one of claims 1 to 6 or claims 7 to 11.

20. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the computer program is executed by a computer, the method described in any one of claims 1 to 6 or claims 7 to 11 is implemented.

Images