Multi-device collaborative control method, graphical interface, and electronic device

Abstract

The present application provides a multi-device collaborative control method, a graphical interface, and an electronic device. The multi-device collaborative control method provided in the present application comprises: a first device displaying a first interface, wherein the first interface comprises a device switching region, an application switching region, and an application control region; in response to a first operation, displaying, in the device switching region, an identifier of a target device after switching, and further displaying, in the application switching region, identifiers of applications of the target device, and displaying, in the application control region, a control panel for the applications of the target device; in response to a second operation, displaying, in the application switching region, an identifier of a target application after switching, and further displaying, in the application control region, a control panel for the target application; and in response to a third operation, controlling the target device to execute a corresponding task by means of the target application.

Description

Multi-device collaborative control method, graphical interface, and electronic device Technical Field

[0001] The present application relates to the field of terminals, and in particular to a multi-device collaborative control method, a graphical interface, and an electronic device. Background Art

[0002] With the development of terminal technology, users can use the applications and functions of electronic devices to carry out activities such as exercise, speech, housework, and entertainment. However, during these activities, users often need to repeatedly pause the current activity because they are far away from the electronic device or their hands are tied, and then move closer to the electronic device to continue the current activity. In addition, when controlling electronic devices to perform multiple tasks, there are also problems such as cumbersome user operations, scattered operation access points, poor human-computer interaction performance, and poor user experience.

[0003] Summary of the Invention

[0004] The present application provides a multi-device collaborative control method, a graphical interface, and an electronic device. The first device displays a first interface, which includes a device switching area, an application switching area, and an application control area. In response to a first operation, the identifier of the target device after switching is displayed in the device switching area, and the identifier of the target device's application is also displayed in the application switching area, and the control panel of the target device's application is displayed in the application control area. In response to a second operation, the identifier of the target application after switching is displayed in the application switching area, and the control panel of the target application is also displayed in the application control area. In response to a third operation, the target device is controlled to perform a corresponding task through the target application.

[0005] In the first aspect, the present application provides a multi-device collaborative control method, which is applied to a first device, and the method includes: displaying a first interface, the first interface including a first area, a second area and a third area; the first area includes an identifier of the second device, the second area includes an identifier of a first group of applications of the second device, and the third area includes a control panel of the first application in the first group of applications; in response to a first operation, displaying the identifier of the switched third device in the first area, displaying the identifier of the second group of applications of the switched third device in the second area, and displaying the control panel of the second application in the second group of applications in the third area; in response to a second operation, displaying the identifier of the switched third application in the second area, and displaying the control panel of the switched third application in the third area; in response to the third operation, executing the first task corresponding to the third application.

[0006] After implementing the method described in the first aspect, you can freely switch and select the target application of the target device among multiple devices and multiple applications by operating the global control interface, and control the running status of the target application in the target device across devices. The global control interface centrally sets up multiple user operation entrances, simplifies the user's operations of switching devices, switching applications, and controlling applications, improves human-computer interaction performance, and enhances user experience.

[0007] In combination with the multi-device collaborative control method described in the first aspect, in response to the second operation, the method further includes: switching the identifier of the application in the selected state in the second area from the identifier of the second application to the identifier of the third application.

[0008] In this way, the user can be further reminded of the target application that has been selected, helping the user to correctly select the desired target application.

[0009] In combination with the multi-device collaborative control method described in the first aspect, the first task includes any one or more of the following: displaying the running details of the third application; or controlling the running status of the third application.

[0010] In this way, in the global control scenario, the first device can not only view the running details of the target application in the target device, but also control the running status of the target application in the target device through corresponding controls.

[0011] In combination with the multi-device collaborative control method described in the first aspect, controlling the running status of the third application includes any one or more of the following: controlling the running status of the third application in the third device, or controlling the running status of the third application in the first device.

[0012] In this way, in the global control scenario, if the target application is not installed on the first device, the target application in the target device can be controlled across devices to perform corresponding tasks. If the target application is installed on the first device, the target application can be run in relay and the target application on the first device can be controlled to perform corresponding tasks.

[0013] In combination with the multi-device collaborative control method described in the first aspect, the third application is an audio and video application, and executing the first task corresponding to the third application includes: controlling the third device to pause, continue or switch the audio and video playback.

[0014] In combination with the multi-device collaborative control method described in the first aspect, the third application is a shooting application, and the first task corresponding to the third application includes: controlling the third device to shoot images.

[0015] In combination with the multi-device collaborative control method described in the first aspect, the third application is an office application, and executing the first task corresponding to the third application includes: controlling the third device to pause, continue or switch the presentation document.

[0016] In combination with the multi-device collaborative control method described in the first aspect, the third application is a social application, and executing the first task corresponding to the third application includes: answering / hanging up the phone.

[0017] In this way, the first device can control the target application in the target device across devices in various application scenarios to perform corresponding tasks, thereby achieving convenient multi-device collaborative control.

[0018] In combination with the multi-device collaborative control method described in the first aspect, before the first operation, the method also includes: in response to a fourth operation, executing a first task corresponding to the first application.

[0019] In combination with the multi-device collaborative control method described in the first aspect, the control panel includes: application information and controls for controlling applications.

[0020] In combination with the multi-device collaborative control method described in the first aspect, the first operation includes any one or more of the following: an operation acting on the first area; or, when the first device is a watch, an operation acting on the crown of the watch.

[0021] In combination with the multi-device collaborative control method described in the first aspect, the second operation includes any one or more of the following: an operation acting on the second area; or, when the first device is a watch, an operation acting on the crown of the watch.

[0022] In combination with the multi-device collaborative control method described in the first aspect, the third operation includes any one or more of the following: operations acting on the third area; or, when the first device is a watch, operations acting on the crown of the watch.

[0023] In this way, the first device can provide the user with virtual controls or physical controls for the user to achieve global control, providing operational diversity and convenience.

[0024] In combination with the multi-device collaborative control method described in the first aspect, the area of ​​the first region is smaller than the area of ​​the second region, and the area of ​​the second region is smaller than the area of ​​the third region.

[0025] In this way, the display size of each area can be set according to the frequency of interaction with the information displayed in different areas, the amount of content, etc. For example, the third area for displaying the application control panel can be set to a larger size, the second area for displaying the application logo can be set to a medium size, and the first area for displaying the device logo can be set to a smaller size.

[0026] In combination with the multi-device collaborative control method described in the first aspect, the first area is located on the upper side of the first device screen, the second area is located on the left side of the first device screen, and the third area is located on the right side of the first device screen.

[0027] In this way, the display position of each area can be set according to the interaction frequency, amount of content, etc. of the information displayed in different areas, which not only improves the aesthetics of the interactive interface but also improves the convenience of interaction.

[0028] In combination with the multi-device collaborative control method described in the first aspect, the first device is a wearable device, such as a watch, so that the user can control the running status of the target application in the target device across devices at any time through the watch.

[0029] In combination with the multi-device collaborative control method described in the first aspect, the method also includes: before displaying the first interface, the first device establishes a connection with the second device.

[0030] In combination with the multi-device collaborative control method described in the first aspect, the first device establishing a connection with the second device specifically includes: displaying a second interface, the second interface including an identifier of a global control function and an identifier of the second device; in response to the operation of dragging the global control function identifier to the identifier of the second device and releasing it, the first device establishing a connection with the second device.

[0031] In this way, the user's operation of selecting the target device can trigger the first device to establish a connection with the target device, thereby realizing subsequent cross-device control functions based on the connection.

[0032] In combination with the multi-device collaborative control method described in the first aspect, the method also includes: before or after displaying the identifier of the third device, the first device establishes a connection with the third device.

[0033] In this way, after selecting a target device or turning on the cross-device control function, connections can be established in advance with multiple nearby devices, so that after the user selects a target device, information of the target application in the target device can be quickly output.

[0034] In a second aspect, the present application provides an electronic device, which includes one or more memories and one or more processors; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute a method as described in any one of the first aspects.

[0035] In a third aspect, the present application provides a communication system, which includes a first device, a second device and a third device, the first device including one or more memories and one or more processors; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the first device to execute the method described in any one of the first aspects.

[0036] In a fourth aspect, the present application provides a chip, which is applied to an electronic device. The chip includes one or more processors, which are used to call computer instructions to enable the electronic device to execute any method described in the first aspect.

[0037] In a fifth aspect, the present application provides a computer-readable storage medium comprising instructions, which, when executed on an electronic device, enable the electronic device to execute the method as described in any one of the first aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] FIG1 is a schematic diagram of a communication system 10 provided in an embodiment of the present application;

[0039] FIG2 is a schematic diagram of the hardware architecture of an electronic device 100 provided in an embodiment of the present application;

[0040] FIG3 is a schematic diagram of a group of enabling multi-device control functions provided by an embodiment of the present application;

[0041] FIG4 is a schematic diagram of an operation interface for using a multi-device collaboration function in a cooking scenario provided by an embodiment of the present application;

[0042] FIG5 is a schematic diagram of an operation interface for using a multi-device collaboration function in a set of fitness scenarios provided by an embodiment of the present application;

[0043] FIG6 is a schematic diagram of an operation interface for using a multi-device collaboration function in a set of camera scenes provided by an embodiment of the present application;

[0044] FIG7 is a schematic diagram of an operation interface for using a multi-device collaboration function in a screen projection presentation scenario provided by an embodiment of the present application;

[0045] Figure 8 is a schematic diagram of the operating interface for using the multi-device collaboration function in a set of movie viewing scenarios provided by an embodiment of the present application. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present application will be described clearly and in detail below with reference to the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In the following, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as implying or suggesting relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more.

[0047] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0048] The term "user interface (UI)" in the following embodiments of this application refers to a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form that the user can receive. The user interface is a source code written in a specific computer language such as Java and extensible markup language (XML). The interface source code is parsed and rendered on an electronic device and finally presented as content that the user can recognize. The commonly used form of user interface is graphical user interface (GUI), which refers to a user interface related to computer operations that is displayed in a graphical manner. It can be a visual interface element such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. displayed on the display screen of an electronic device.

[0049] When users use the applications and functions of electronic devices to carry out activities such as housework, exercise, work, and entertainment, in order to provide users with an uninterrupted and immersive activity experience, this application provides a multi-device collaborative control method, a graphical interface, and an electronic device.

[0050] In the multi-device collaborative control method provided in the present application, the first device displays a first interface, which includes a device switching area, an application switching area, and an application control area. In response to the first operation, the identifier of the target device after switching is displayed in the device switching area, and the identifiers of a group of applications of the target device are also displayed in the application switching area, and the control panel of the target application in the group of applications is displayed in the application control area. In response to the second operation, the identifier of the target application after switching is displayed in the application switching area, and the control panel of the target application after switching is also displayed in the application control area. In response to the third operation, the first device performs tasks corresponding to the target application, such as viewing the running details of the target application, controlling the running status of the target application in the target device across devices, etc.

[0051] After implementing the multi-device collaborative control method, graphical interface and electronic device provided in this application, global control of multiple devices, multiple applications, and the operating status of each application can be achieved. In addition, the user operation entrance is centralized, which simplifies user operations, improves human-computer interaction performance, and enhances user experience.

[0052] Next, we briefly introduce the relevant concepts involved in this application.

[0053] The multi-device collaborative control function refers to two or more electronic devices working together or collaboratively to perform certain tasks, such as performing application relay, content sharing, capability sharing, etc. across devices through multi-device collaboration.

[0054] Application relay: An electronic device can relay the running of an application running on another electronic device. For example, the first device can relay the running of the target application based on the received running data of the target application in the target device sent by the target device. In addition, the first device can also perform tasks corresponding to the target application in response to operations acting on the target application, such as viewing the running details of the target application, controlling the running status of the target application in the target device across devices, and so on. In the process of the first device relaying the running of the application of the target device, in one feasible way, the first device may not actually run the application, but display the application status and related controls according to the running data of the application transmitted by the target device; in another feasible way, if the application is installed in the first device, the first device may also run the application, rather than just displaying the application status and related controls according to the running data of the application transmitted by the target device. In addition, in the process of the first device taking over the operation of the application in the target device, in one feasible manner, the target device may continue to run the application. When the target device continues to run the application, the first device can control the state of the target device running the application across devices; in another feasible manner, the target device may also stop running the application. When the target device stops running the application, the first device can continue to run the application and control the running state of the application on the first device.

[0055] Content sharing: Sharing files, photos, videos, audio, text, and other content between different electronic devices. For example, a first device can receive content shared by a target device or share content with a target device.

[0056] Capability sharing: Sharing each other's software and hardware capabilities between different electronic devices. For example, taking the shared screen display capability as an example, the target device equipped with a larger display screen not only provides content display function for itself, but also provides content display function for the first device, that is, the first device can project its own content to the target device for display and control. Taking the shared sensor hardware capability as an example, the first device equipped with a sensor not only provides sensor data detection function for itself, but also provides sensor detection function for the target device, that is, the target device can control the first device to detect and obtain sensor data, etc. Cameras, cellular networks, etc. can also be shared.

[0057] Based on the above brief introduction to the multi-device collaborative control function, we can see that the application relay, content sharing, and capability sharing it provides all involve cross-device control. Therefore, in cross-device control scenarios, especially those involving cross-device control of multiple devices and multi-application collaborative execution of multiple tasks, only better global control can provide better multi-device collaborative control functions.

[0058] The first device refers to a device that plays the role of a control center in a multi-device collaborative control scenario, and therefore, it can also be called a central device.

[0059] In an optional embodiment, the first device includes any device in a multi-device collaborative control scenario, such as a mobile phone, a laptop, a tablet, etc. shown in FIG1 described later. This is because, generally, each device in a multi-device collaborative control scenario has the ability to control other devices. When performing different cross-device tasks, each device may act as a central device to control other devices.

[0060] In a preferred embodiment, the first device includes a wearable device in a multi-device collaborative control scenario. This is because wearable devices have the characteristics of portability and lightness, which can facilitate users to freely and quickly control the execution of multi-device collaborative control tasks in various activity scenarios such as sports, work, housework, and entertainment, without interrupting the user's ongoing activities, giving users a smooth and immersive activity experience. Among them, wearable devices mostly exist in the form of portable accessories, and specific product forms include but are not limited to: products supported by the wrist such as watches, bracelets, etc., and products supported by the head such as glasses, helmets, headbands, etc., and the embodiments of this application do not impose specific restrictions on this.

[0061] The target device refers to the device that plays the role of being controlled in a multi-device collaborative control scenario.

[0062] In an optional embodiment, the target device includes any device in a multi-device collaborative control scenario. This is because each device in a multi-device collaborative control scenario supports being controlled by other devices, and when executing different cross-device tasks, each device may serve as a controlled device.

[0063] connect

[0064] Number of connections: The first device can support simultaneous connection to one or more devices, that is, the devices connected to the first device include but are not limited to connecting to the second device.

[0065] Connection time: The first device may establish a connection with the second device before receiving the first operation, or may establish a connection with the second device after receiving the first operation.

[0066] Connection Type: A first device can establish a connection with another device, such as a second device or a third device, via wireless local area networks (WLAN), Bluetooth (BT), near field communication (NFC), or cellular communication. This connection can be short or long. The terms "long" and "short" are relative terms. A short connection means that the first and second devices establish a connection only when data is needed and disconnect after the data transfer is complete. Each short connection is used only to complete a single data transfer. A long connection means that the first and second devices maintain the connection after data transfer is complete, without disconnecting. If further data transfer is needed, data can be transferred directly over the maintained connection. When a long connection is maintained, the first and second devices can send multiple data packets continuously over the same connection. Furthermore, during a long connection, if no data packets are sent for an extended period, the first and second devices can perform a communication link test to promptly detect and correct any communication link failures.

[0067] operate

[0068] First operation: used to control the first device to switch the target device from one or more nearby devices, for subsequent cross-device control of the target device to perform tasks, and in response to the first operation, the first device also displays the identifier of the switched target device. Specifically, the first device may display the identifiers of one or more devices in the first interface. If the identifier of one device is displayed, the device is the target device; if the identifiers of multiple devices are displayed, the identifier of the device in the selected state is the identifier of the target device. The first operation includes any of the following types of operations: sliding or clicking operations on the device switching area, or operations on the physical controls of the first device, etc. When the first device is a watch, the physical control includes a crown.

[0069] The second operation is used to control the first device to switch the target application from a group of applications on the target device, and to subsequently control the target application of the target device across devices to perform tasks. In response to the second operation, the first device also displays the identifier of the switched target application. Specifically, the first device may display one or more application identifiers in the first interface. If one application identifier is displayed, the application is the target application; if multiple application identifiers are displayed, the identifier of the selected application is the identifier of the target application. The second operation includes any of the following types of operations: sliding or clicking operations on the application switching area, or operations on the physical controls of the first device. When the first device is a watch, the physical control includes a crown.

[0070] The third operation is used to view the running details of the target application, or to control the target application of the target device across devices to perform tasks. The implementation of the third operation may vary for different types of target applications. For details, please refer to the several different multi-device collaborative control scenarios introduced later. I will not go into details here.

[0071] Task

[0072] The first device performs a task corresponding to the target application (also referred to as a first task), including any one or more of the following tasks: displaying the running details of the target application, or controlling the running status of the target application. Controlling the running status of the target application includes: controlling the running status of the target application on the target device across devices, and / or controlling the running status of the target application (if any) on the first device.

[0073] FIG1 is a communication system 10 used in the multi-device collaborative control method provided in this application.

[0074] As shown in FIG1 , the communication system 10 includes, but is not limited to, a watch, a mobile phone, a notebook, a tablet, a smart screen, a treadmill, or a projection device. The watch can be referred to as the first device, and the mobile phone, notebook, tablet, smart screen, treadmill, and projection device are all connectable devices near the watch. These devices can all serve as target devices of the first device, and the first device can control the target device to perform a task through cross-device communication.

[0075] The connection methods between the devices in the communication system 10 include, but are not limited to, any one or more of the following: connections established via WLAN, BT, NFC, or cellular communication. The connections established via WLAN and BT can be point-to-point, point-to-multipoint, or multipoint-to-multipoint, that is, the watch can establish connections with one or more devices at the same time, which is not limited in the present embodiment.

[0076] The communication system 10 can provide multi-device collaborative control functions, such as the aforementioned multi-device collaborative execution of cross-device application relay, content sharing, capability sharing, and other tasks. In addition, the global control interface provided by the device serving as the control center can be used to control multiple devices and multiple applications to perform corresponding cross-device tasks. For example, taking a watch as the control center, the watch can establish a connection with any one or more devices in a nearby mobile phone, laptop, tablet, smart screen, treadmill, and projection device. The watch can display a global control interface, which includes but is not limited to a device switching area, an application switching area, and an application control area. The user can operate the global control interface to control the devices near the watch to perform corresponding tasks across devices. Specifically, in response to a first operation on the watch, the watch can switch to a different target device from a nearby device for subsequent cross-device control of the target device to perform a task. After the watch switches the target device, in response to a second operation on the watch, the watch can also switch to a different target application from a group of applications on the target device for subsequent cross-device control of the target application in the target device to perform a task. After the watch switches the target application, in response to the operation on the target application, the watch can view the running details of the target application or control the running status of the target application across devices. For the specific implementation of the multi-device collaborative control function provided by the communication system 10, please refer to the detailed description of the UI implementation later, which will not be repeated here.

[0077] The types and quantities of electronic devices included in the communication system 10 shown in FIG1 are only examples, and the present application does not impose any restrictions on the types and quantities of electronic devices included in the communication system 10. Or terminal devices of other operating systems, such as wearable devices, mobile phones, tablet computers, desktop computers, laptop computers, handheld computers, notebook computers, ultra-mobile personal computers (UMPC), netbooks, as well as cellular phones, personal digital assistants (PDAs), augmented reality (AR) devices, virtual reality (VR) devices, artificial intelligence (AI) devices, vehicle-mounted devices, smart home devices and / or smart city devices, etc., the present application does not limit the specific form of the electronic device. Optionally, in most scenarios, the electronic device that plays the role of the control center can be any type of device such as a watch, a mobile phone, a notebook or a tablet, etc. Preferably, in activity scenarios such as sports, work, housework, and entertainment, the electronic device that plays the role of the control center can be limited to a wearable device, so that the user can conveniently and quickly operate the wearable device to perform tasks across devices, etc. Figure 1 only uses a watch as an example of the control center, which should not constitute a limitation to the present application.

[0078] Based on the above introduction to the communication system 10 , the electronic devices involved in the communication system 10 will be described in detail below with reference to FIG. 2 .

[0079] The electronic device 100 shown in FIG2 can be any electronic device included in the communication system 10. The electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a sensor module 180, a physical control 190, a camera 193, a display screen 194, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an acceleration sensor 180E, a touch sensor 180K, etc.

[0080] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0081] The processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units can be independent devices or integrated into one or more processors. Among them, the controller can be the nerve center and command center of the electronic device 100. The controller can generate an operation control signal based on the instruction opcode and the timing signal to complete the control of instruction fetching and execution.

[0082] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0083] In an embodiment of the present application, the processor 110 may be used to cooperate with other software and hardware modules of the electronic device 100 to execute the following method: based on the detected operation, enable the multi-device collaborative control function, and after enabling the multi-device collaborative control function, switch the target device to be controlled based on the received first operation, switch the target application to be controlled based on the received second operation, and execute tasks related to the target application based on the received third operation. For the specific implementation of the above method, please refer to the detailed description of the UI embodiment below, which will not be repeated here.

[0084] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0085] The USB interface 130 is an interface that complies with USB standards and may be a Mini USB interface, a Micro USB interface, a USB Type-C interface, or the like. The USB interface 130 can be used to connect a charger to charge the electronic device 100, or to transfer data between the electronic device 100 and peripheral devices. It can also be used to connect headphones to play audio. This interface can also be used to connect other electronic devices, such as augmented reality devices.

[0086] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0087] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0088] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0089] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0090] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker, a receiver, etc.) or displays an image or video through the display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0091] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, demodulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0092] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with a network and other devices through wireless communication technologies. The wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0093] In an embodiment of the present application, the electronic device 100 can couple the antenna 1 and the mobile communication module 150, or couple the antenna 2 and the wireless communication module 160, so that the electronic device 100 can establish a connection and transmit data with the network and other devices through wireless communication technology.

[0094] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0095] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD). The display screen panel can also be made of an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniLED, a microLED, a micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 can include one or N display screens 194, where N is a positive integer greater than one.

[0096] In an embodiment of the present application, the electronic device 100 can display a global control interface related to the multi-device collaborative control function through the display screen 194, including but not limited to the various user interfaces shown in Figures 4 to 8 below.

[0097] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0098] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and converted into a visible image. The ISP can also perform algorithmic optimization on image noise and brightness. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0099] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0100] In an embodiment of the present application, the electronic device 100 can perform a shooting task after receiving a cross-device instruction from another device. For example, the tablet described later can perform a video recording task after receiving a cross-device instruction from a watch.

[0101] The internal memory 121 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).

[0102] Random access memory (RAM) may include static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, for example, the fifth generation of DDR SDRAM is generally referred to as DDR5 SDRAM), etc. The RAM can be directly read and written by the processor 110 and can be used to store executable programs (e.g., machine instructions) of the operating system or other running programs, as well as user and application data.

[0103] The non-volatile memory may include a disk storage device and a flash memory. The non-volatile memory may also store executable programs and user and application data, etc., and may be pre-loaded into the random access memory for direct reading and writing by the processor 110.

[0104] The external memory interface 120 can be used to connect to an external non-volatile memory to expand the storage capacity of the electronic device 100. The external non-volatile memory communicates with the processor 110 via the external memory interface 120 to implement data storage. For example, files such as music and videos can be stored in the external non-volatile memory.

[0105] In an embodiment of the present application, the electronic device 100 may store information on applications for cross-device control of a target device. For example, the watch described later stores information on the running status of applications A, B, and C in a mobile phone and corresponding control functions, stores information on the running status of applications X, Y, and Z in a tablet and corresponding control functions, stores information on the running status of applications L, M, and N in a tablet and corresponding control functions, or also stores information on the running status of applications U, V, and W in a smart screen and corresponding control functions.

[0106] The electronic device 100 can implement audio functions through the audio module 170, speakers, receivers, microphones, headphone jacks, and application processors. For example, music playback, recording, etc. The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be provided in the processor 110, or some functional modules of the audio module 170 can be provided in the processor 110.

[0107] In an embodiment of the present application, when the electronic device 100 takes over the running of applications in other devices, it can continue to play the running status of applications in other devices through the audio module 170. For details, please refer to the watch continued to play music in the mobile phone, recipes in the tablet, etc. introduced later.

[0108] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A can be located on display screen 194. There are many types of pressure sensors 180A, such as resistive, inductive, and capacitive. A capacitive pressure sensor can include at least two parallel plates made of conductive material. When force is applied to pressure sensor 180A, the capacitance between the electrodes changes. Electronic device 100 determines the intensity of the pressure based on this change in capacitance. When a touch operation is applied to display screen 194, electronic device 100 detects the intensity of the touch operation based on pressure sensor 180A. Electronic device 100 can also calculate the location of the touch based on the detection signal from pressure sensor 180A. In some embodiments, touch operations applied to the same touch location but with different touch intensities can correspond to different operation instructions. For example, when a touch operation with an intensity less than a first pressure threshold is applied to a short message application icon, an instruction to view short messages is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to a short message application icon, an instruction to create a new short message is executed.

[0109] The gyroscope sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyroscope sensor 180B can also be used for navigation and somatosensory game scenes.

[0110] Accelerometer 180E can detect the magnitude of acceleration of electronic device 100 in all directions (generally three axes). It can also detect the magnitude and direction of gravity when electronic device 100 is stationary. It can also be used to identify the electronic device's posture, enabling applications such as switching between landscape and portrait modes and pedometers.

[0111] The touch sensor 180K is also called a "touch panel." The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen." The touch sensor 180K is used to detect touch operations applied thereto or in the vicinity thereof. The touch sensor can transmit the detected touch operations to the application processor to determine the type of touch event. Visual output related to the touch operations can be provided via the display screen 194. In other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 100, in a location different from that of the display screen 194.

[0112] In the embodiment of the present application, the electronic device 100 can detect the operation acting on the corresponding control in the display screen 194 through the touch sensor 180K, and then perform the corresponding task in response to the operation.

[0113] Physical control keys 190 include a power button, volume buttons, and a rotary control (applied to a watch). Physical control keys 190 can be mechanical buttons or touch buttons. Electronic device 100 can receive key inputs and generate key signal inputs related to user settings and function control of electronic device 100.

[0114] Based on the communication system 10 and electronic device 100 introduced above, the specific implementation of the multi-device collaborative control function provided by this application is introduced in conjunction with Figures 3-4, and how to apply the multi-device collaborative function in various scenarios is introduced in conjunction with Figures 5-8.

[0115] FIG3 exemplarily shows a set of operation interface diagrams for enabling the multi-device collaborative control function in the first device.

[0116] Figure 3a shows a user interface 31 of the first device. This user interface 31 can be a settings interface of the first device or a control center interface opened by a pull-down operation, where the pull-down operation can be an operation in which a user swipes down from the top of the watch face. This user interface 31 includes, but is not limited to, options 311 for the collaborative control function, options 312 for the wireless local area network, and options 313 for Bluetooth.

[0117] While the first device displays "a" in FIG3 , the user can enable / disable the corresponding function by inputting operations on each function option. For example, after the user clicks option 312 and option 313, the first device enables functions such as wireless LAN and Bluetooth, and also adjusts option 312 and option 313 to the enabled state as shown in "a" in FIG3 .

[0118] FIG3 b shows a user interface 32 of the first device. The user interface 32 may be the details page of the collaborative control function that is opened after the first device receives an operation on option 311. The user interface 32 includes: an identifier of the first device and identifiers of nearby devices. The identifier of the first device is the identifier 321 of the local device, and the identifiers of the nearby devices include the identifier 322 of a mobile phone, the identifier 323 of a tablet, the identifier 324 of a laptop, and the identifier 325 of a smart screen. The display format of the identifier 321 of the local device and the identifiers of the nearby devices may be different. For example, the background fill color of the identifier of the local device is extracted from the wallpaper of the local device, and the area occupied by the identifier of the local device on the display interface is larger than the area occupied by the identifier of the nearby device on the display interface. The identifiers of the nearby devices are queried by the first device based on communication methods such as WLAN, Bluetooth, NFC, scanning a QR code, or a wired connection. The identifiers of the aforementioned devices include but are not limited to the device name, device model, or device icon, etc. FIG3 only uses the device name as an example of the device identifier, and this embodiment of the application is not limited to this.

[0119] When the first device displays b in FIG. 3 , the user can click on the local device's logo 321 to expand the local device's advantageous services, thereby enabling subsequent users to drag the required services to the target device's logo and release them to transfer the services across devices.

[0120] Figure 3 c is a user interface 33 shown on the first device. The user interface 33 may be another detail page of the collaborative control function that is opened after the first device receives an operation of clicking on the identifier 321 of the local device. The user interface 33 includes the identifier of the first device, i.e., the identifier 331 of the local device, and the identifiers of the nearby devices such as the identifier 332 of the mobile phone, the identifier 333 of the tablet, the identifier 334 of the notebook, and the identifier 335 of the smart screen, etc. Optionally, the display form of the identifier 331 of the local device here is different from that of the identifier 321 of the previous text machine. For example, the periphery of the identifier 331 of the local device here can display an effect that matches the (Industrial design, ID) design of the local device. In addition, the identifier 331 of the local device also includes identifiers corresponding to the advantage services in the local device, such as the identifier 331A of the global control, the identifier 331B of the sports health, or the identifier 331C of the screen. The aforementioned advantage services refer to software and hardware functional modules that are not available in other devices, or refer to software and hardware functional modules that can bring unique experience to users when running in the first device. Taking global control as an example, the advantage service can provide users with a centralized control entrance to control other nearby devices to perform corresponding tasks; taking sports health as an example, since the first device has a variety of sensors that fit the user's body, the advantage service can provide users with more accurate and comprehensive health data and share the health data with other nearby devices; taking the screen as an example, since the first device can be portable, the screen display information of other devices is shared in the first device, and the advantage service can provide users with an experience that is convenient for viewing information on other devices; or, the screen information of the first device is shared in other devices, and the advantage service can provide users with the experience of viewing information on the first device with the help of other devices with larger screens. The embodiments of this application do not impose specific restrictions on the types and quantities of advantage services. Only a few are shown here as examples, which should not constitute a limitation on this application.

[0121] In the embodiment of the present application, the above-mentioned user interface 33 can also be referred to as a second interface.

[0122] During the process of the first device displaying c of Figure 3, the user can select the logo of the target service from the logo of the first device's advantage service, drag it to the logo of the target device and release it, thereby triggering the flow of the service between the first device and the target device. In addition, at different processing stages, the position and size of the hot zone where the local logo is located may change. For example, before clicking the local logo 321 shown in b of Figure 3, the hot zone where the local logo 321 is located is in the center of the interface and is small in size. After clicking the local logo 321 shown in b of Figure 3, the local logo 321 becomes the local logo 331 shown in c of Figure 3, and the size of the hot zone where the local logo 331 is located will increase as the advantage service is expanded. After dragging the logo of the advantage service shown in c of Figure 3, the size of the hot zone where the local logo 331 is located will decrease as the advantage service is detached. The embodiment of the present application does not specifically limit the size and position of the hot zone where the local logo is located. The embodiment of the present application does not specifically limit the size and position of the hot zone where the local logo is located.

[0123] Figure 3d shows a user interface 34 on the first device. This user interface 34 may be the global control interface displayed after the first device receives an action to drag and drop the global control icon 331A to the phone icon 332. This user interface 34 includes, but is not limited to, a device switching area 341, an application switching area 342, and an application control area 343.

[0124] Among them, the device switching area 341 displays the target device identifier 341A, as well as controls 341B and 341C for switching the target device. The target device identifier 341A is the identifier of the mobile phone selected in FIG3 b. Controls 341B and 341C can support user clicks. In response to the click operation, the first device switches the target device from other nearby devices. Optionally, the device switching area 341 can also display the identifiers of multiple devices. Only the device corresponding to the identifier in the selected state is the target device, and the device corresponding to the identifier in the unselected state is the other device near the first device.

[0125] In addition, after the first device turns on the collaborative control function, the first device can establish a connection only with the target device, or the first device can also establish a connection with the target device and other nearby devices that have served as target devices within a preset period of time. This allows the target device to be quickly controlled to perform cross-device tasks when the target device is subsequently switched from a nearby device.

[0126] Among them, a group of application identifiers are displayed in the application switching area 342. The identifiers of this group of applications are the identifiers of one or more applications previously selected from the application identifiers of the target device, such as the identifiers of application A, application B, and application C, etc. The embodiment of the present application does not limit the number of applications and the display form of the identifiers. The identifiers of different applications will vary with the characteristics of different applications. When the number of applications is greater than the preset number (for example, 3), the application switching area 342 only displays the identifiers of 3 of the applications. The user can switch the display of the identifiers of the remaining undisplayed applications by sliding the application switching area or operating the physical controls (such as rotating the crown). The application corresponding to the identifier in the selected state in the application switching area 342 is the target application. The selected state can specifically be a state in which the icon size of the application becomes larger, or a state in the center area of ​​the application switching area 342, etc. The present application does not limit the specific display form of the selected state.

[0127] Optionally, after the first device selects the target device and before switching to the target application, the selected target application may default to any one or more of the following: the application currently running in the foreground of the target device, a historically frequently used application in a multi-device collaborative control scenario, or the first application dragged from the application of the target device to the target area, etc., and the present embodiment does not impose any restrictions on this. For example, if the target device is running application A in the foreground to play music, the center area of ​​the application switching area 342 is enlarged and displays the logo of application A.

[0128] The application control area 343 displays the control panel of the target application. This control panel includes, for example, the target application's operating status 343A and controls 343B for viewing the target application's operating details. Specifically, if the target device is currently running Application A and playing music provided by Application A, after the first device establishes a connection with the target device and selects Application A as the target application, the first device displays Application A's control panel based on the operating data of Application A transmitted by the target device. It is understood that the content contained in the control panel may vary for different target applications. For example, a music application's control panel includes, but is not limited to, the title and artist name of the currently playing music, controls for switching songs, controls for starting / stopping songs, and controls for viewing music details. If a user wishes to control Application A to start / stop / switch songs, they can input operations into the corresponding controls in the application control area 343, thereby causing the first device to control Application A on the target device to start / stop / switch songs across devices. If a user wishes to view the operating details of Application A, they can input operations into the corresponding controls in the application control area 343, thereby causing the first device to display detailed information such as the lyrics of the song played by Application A.

[0129] In the embodiment of the present application, the above-mentioned user interface 34 and each user interface shown in Figures 4 to 8 below can be referred to as the global control interface or the first interface. In addition, the device switching area in the first interface can also be referred to as the first area, the application switching area can also be referred to as the second area, and the application control area can also be referred to as the third area.

[0130] During the process of the first device displaying d in Figure 3, the user can globally control multiple devices, multiple applications, and multiple tasks through the user interface 34. The global control includes but is not limited to: the user can switch the target device in the device switching area 341, and switch the target application in the application switching area 342, and input the corresponding operation (also known as the fourth operation) into the control panel to control the first device to execute the task corresponding to the target application (also known as the first task).

[0131] Next, we will introduce in detail how to use the multi-device collaboration function through global control in combination with various application scenarios.

[0132] Scenario 1: A user uses a tablet's recipe app to perform household chores.

[0133] Specifically, when performing household chores such as cooking, a user may need to use a recipe application to guide the user on how to cook, or may need a music application to play music. Although it is very likely that the user will not be able to stop what they are doing to approach the target device and control the target device to play recipes or music, by using the multi-device collaborative control function of the first device to control the tablet to play recipes and the mobile phone to play music across devices, it is convenient for the user to freely control the corresponding electronic devices to play recipes and music while cooking, thereby ensuring that the user can cook smoothly according to the instructions of the recipe.

[0134] FIG4 exemplarily shows a schematic diagram of an operation interface for using a multi-device collaboration function in a cooking scenario.

[0135] Figure a in Figure 4 shows a user interface 41 for a first device to control application A on target device 1 across multiple devices. User interface 41 includes a device switching area 411, an application switching area 412, and an application control area 413. For example, target device 1 is the mobile phone shown in Figure 3 . User interface 41 is similar to user interface 34 in Figure 3 . For details on how the first device enables the multi-device collaborative control function, selects target device 1, and then performs cross-device control of target device 1 to play music, please refer to the description of Figure 3 above. The origin of user interface 41 will not be elaborated here.

[0136] After the first device displays the control interface of application A in target device 1, that is, after the first device displays a in FIG4 , the first device may perform any one or more of the following:

[0137] (1) The first device can control the running status of application A in the target device 1 across devices, for example, by operating the corresponding controls in the application control area 413 in Figure 4a, to control the target device 1 to stop / start / switch playing music, etc.

[0138] (2) If the user wants to control an application in another device to perform a task through the first device, the user can control the first device to switch target device 1 to target device 2 by any of the following operations: clicking control 411B or control 411C in FIG4 a, sliding left / right on marker 411A, or selecting device switching area 411 and then operating the physical control of the first device (for example, rotating the crown of the first device). The embodiment of the present application does not impose any specific restrictions on the triggering operation for controlling the first device to switch the target device.

[0139] (3) The user can view the information shown in the running status 413A of the target application in the application control area 413, and the user can input corresponding operations to the control 413B in the application control area 413 to control the first device to execute the task corresponding to the target application.

[0140] In Figure 4 , b and d are user interfaces for the first device to control target device 2 across devices. Specifically, b in Figure 4 is the initial user interface 42 for the first device to globally control target device 2 after switching from target device 1 to target device 2. c in Figure 4 is the user interface (control interface) 43 for the first device to display application X on target device 2. d in Figure 4 is the user interface 44 for the first device to control application X on target device 2 across devices. The details are as follows:

[0141] The user interface 42 shown in b of Figure 4 is an interface displayed after the first device receives an operation for switching to the target device 2. The user interface 42 includes a device switching area 421, an application switching area 422, an application control area 423, and the like.

[0142] The device switching area 421 displays a target device identifier 421A, as well as a control 421B for switching to the previous target device and a control 421C for switching to the next target device. Target device identifier 421A is the identifier of target device 2, such as a tablet. Controls 421B and 421C can be clicked by the user. In response to the click, the first device switches the target device from other nearby devices. For a detailed description of the device switching area 421, please refer to the previous description of the device switching area 341, and will not be repeated here.

[0143] Among them, a group of application identifiers are displayed in the application switching area 422, and the identifiers of this group of applications are identifiers corresponding to the applications of the target device 2, for example, including identifiers of application X, application Y and application Z, among which application Y is the default target application. The identifiers of this group of applications can be selected by the user, or provided by the target device 2 by default. The target device 2 can provide the identifier of the application by default according to any one or more of the following rules: the application currently running on the target device, the identifier of the historically frequently used application of the target device, the identifier of the historically frequently used application in the multi-device collaborative control scenario, or the identifier of the application selected last time, etc. For a detailed introduction to the application switching area 422, please refer to the previous description of the application switching area 342, which will not be repeated here.

[0144] The application control area 423 displays a control panel for the target application, including, for example, the target application's operating status. If target device 2 is not currently running application Y, after the first device establishes a connection with target device 2 and selects application Y as the target application, the first device will prompt the user whether to enable application Y on target device 2 across devices. If target device 2 is currently running application Y, the first device will not display a message prompting the user whether to enable application Y on target device 2 across devices, but will instead directly display the operating status of application Y and related controls.

[0145] While the first device is displaying portion b of FIG. 4 , if application X is not a recipe application and the user wants to control another application to play a recipe, the user can input a sliding operation into application switching area 422 to switch the target application from application Y to application X that can play recipes.

[0146] 4 c is a control interface of application X displayed after the first device receives an operation to switch the target application from application Y to application X. The user interface 43 includes a device switching area 431 , an application switching area 432 , and an application control area 433 .

[0147] Target device identifier 431A in device switching area 431 displays the same information as target device identifier 421A in device switching area 421, indicating that the target device remains target device 2 (i.e., tablet). Controls 431B and 431C in device switching area 431 have the same functions as controls 421B and 421C in device switching area 421.

[0148] The order of the application identifiers in the application switching area 432 is different from that in the aforementioned application switching area 422. Optionally, when the number of target applications 2 to be controlled is large (e.g., greater than 3), the identifiers of the applications in the application switching area 432 are also different from those in the aforementioned application switching area 422.

[0149] Application control area 433 displays different content than application control area 423. Application control area 433 displays the control panel of the switched application X. If target device 2 is currently running application X and has opened a recipe provided by application X, after the first device establishes a connection with target device 2 and selects application X as the target application, the first device displays the control panel of application X based on the operating data of application X transmitted by target device 2. It will be appreciated that the content contained in the control panel may vary for different target applications. For example, a recipe application control panel includes, but is not limited to, a recipe summary 433A and a control 433B for viewing recipe details.

[0150] After the first device displays the control interface of application X of target device 2, that is, after the first device displays c in FIG. 4 , the first device may perform any one or more of the following tasks corresponding to application X (also referred to as the first task):

[0151] (1) If the user wants to control and view the detailed recipe provided by application X, the user can click control 433B to trigger the first device to display the detailed recipe, which is obtained by the first device from the running data of application X transmitted from target device 2.

[0152] (2) If the user wants to control the target device 2 to perform a task through the first device, the user can click the corresponding control shown in the application control area 433 in Figure 4c to control the target device 2 to stop / start / switch the playing of the recipe, etc. Alternatively, the user can also control the target device 2 to stop / start / switch the playing of the recipe, etc. by selecting the application control area 433 and then operating the physical control of the first device (for example, rotating and pressing the crown of the first device).

[0153] 4 d is the interface displayed by the first device after receiving the recipe playing application X of the cross-device control target device 2. The user interface 44 includes a device switching area 441, an application switching area 442, and an application control area 443.

[0154] Target device identifier 441A in device switching area 441 displays the same information as target device identifier 431A in device switching area 431, indicating that the target device remains target device 2 (i.e., tablet). Controls 441B and 441C in device switching area 441 have the same functions as controls 431B and 431C in device switching area 431.

[0155] The identifiers of the applications in the application switching area 442 and the aforementioned application switching area 432 are the same.

[0156] Application control area 443 displays different content than application control area 433. Recipe overview 443A in application control area 443 displays the updated control panel of application X. Specifically, if target device 2 starts playing a recipe provided by application X, the playback status of the recipe displayed on the first device switches from the previous stopped state to the playing state. Control 443B for viewing recipe details in application control area 443 has the same function as control 433B for viewing recipe details.

[0157] In the scenario shown in Figure 4, target device 1 can also be referred to as the second device, target device 2 can also be referred to as the third device, and the operation for controlling the first device to switch target device 1 to target device 2 can also be referred to as the first operation. Application A can also be referred to as the first application, application Y can also be referred to as the second application, and application X can also be referred to as the third application. The operation for controlling the first device to switch the target application from application Y to application X can also be referred to as the second operation. The operation for controlling the first device to view the detailed recipe provided by application X and the operation for controlling the first device to control application X on target device 2 across devices to perform a task can also be referred to as the third operation.

[0158] Scenario 2: Users use sports apps on their phones to perform fitness activities.

[0159] When users use sports apps on their mobile phones to perform fitness activities, they may need to pause / resume / switch fitness courses played on the target device. Although it is inconvenient for users to stop their current activity to pick up their mobile phone and control the mobile phone to pause / resume / switch the fitness course being played, the method described below in this application for controlling the mobile phone across multiple devices by fully utilizing the multi-device collaborative control function of the first device allows users to freely control the mobile phone to play fitness courses, capture images, etc. during fitness, thereby ensuring that users can smoothly follow the instructions of the fitness course and complete the movements.

[0160] FIG5 exemplarily shows a schematic diagram of an operation interface for using a multi-device collaboration function in a set of fitness scenarios.

[0161] a in Figure 5 is a user interface 51 (control interface) when the first device displays application C in the target device. The user interface 51 may be the interface displayed after the first device turns on the multi-device collaborative control function and sets the target device to a mobile phone and the target application to application C. The user interface 51 may be the interface displayed after the first device displays the interface shown in d in Figure 3 and receives an operation for switching the target application from application A to application C. Regarding the specific implementation method of turning on the multi-device collaborative control function and setting the target device and target application, please refer to the previous description of setting the mobile phone / tablet as the target device and setting application A / application X as the target application in Figures 3-4, which will not be repeated here.

[0162] The user interface 51 displays a device switching area 511, an application switching area 512, an application control area 513, etc. The user interface 51 is similar to the user interface 34 in FIG3, except that the target application is different, and the target application here is application C of the mobile phone.

[0163] The device switching area 511 displays the target device identifier 511A and controls 511B and 511C for switching the target device. Detailed description of the device switching area 511 can be found in the previous description of the device switching area 341, which will not be repeated here.

[0164] The application switching area 512 displays a group of application identifiers corresponding to the applications of the target device, such as identifiers of application A, application B, and application C. Application C is a target application, and the target application may be switched from application A to application C as shown in FIG3 d. For a detailed description of the application switching area 512, please refer to the previous description of the application switching area 342, and will not be repeated here.

[0165] The application control area 513 displays a control panel for the target application. For example, the control panel includes the target application's running status 513A and a view details control 513B. Currently, the target application's running status 513A indicates that the fitness course has been stopped. For a detailed description of the device's application control area 513, please refer to the description of the application control area 343 above, and will not be repeated here.

[0166] After the first device displays the control interface of application C in the target device, that is, after the first device displays a in Figure 5, the first device can control the running status of application C in the target device across devices. For example, by operating the corresponding controls in the application control area 513 in a in Figure 5, the first device can control the target device across devices to stop / start / switch playing fitness courses, etc.

[0167] The user interface 52 shown in FIG5(b) is displayed after the first device receives an operation for controlling the target device to play a fitness course across devices. The operation for playing the fitness course can be clicking the control for playing the next page in the running state 513A shown in FIG5(a), or it can be an operation of a physical control on the first device (such as rotating a crown).

[0168] The user interface 52 displays a device switching area 521 , an application switching area 522 , an application control area 523 , and the like.

[0169] Target device identifier 521A in device switching region 521 displays the same content as target device identifier 511A in device switching region 511, indicating that the target device remains a mobile phone. Controls 521B and 521C in device switching region 521 have the same functions as controls 511B and 511C in device switching region 511.

[0170] The identifiers of the applications in the application switching area 522 and the aforementioned application switching area 512 are the same.

[0171] Application control area 523 displays different content than application control area 513. Specifically, the target application's running status 523A in application control area 523 indicates that the target application is currently playing a fitness class, while the target application in application control area 513 indicates that the fitness class has been stopped. The view details control 523B in application control area 523 has the same function as the view details control 513B.

[0172] When the first device continues to display the control interface of application C of the target device, that is, after displaying the interface shown in FIG5 b, if the user wants to start / stop / switch playing the fitness course, the user can input an operation to the corresponding control in the application control area 523, thereby enabling the first device to control application C of the target device across devices to start / stop / switch playing the fitness course.

[0173] The user interface 53 shown in c of Figure 5 is the interface displayed after the first device receives an operation for pausing the playing of the fitness course. The user interface 53 includes a device switching area 531, an application switching area 532, and an application control area 533.

[0174] Target device identifier 531A in device switching region 531 displays the same content as target device identifier 521A in device switching region 521, indicating that the target device remains a mobile phone. Controls 531B and 531C in device switching region 531 have the same functions as controls 521B and 521C in device switching region 521.

[0175] The identifiers of the applications in the application switching area 532 and the aforementioned application switching area 522 are the same.

[0176] Application control area 533 displays different content than application control area 523. Specifically, the target application's running status 533A in application control area 533 indicates that the target application has stopped playing a fitness class, while the target application in application control area 523 is currently playing a fitness class. The view details control 533B in application control area 533 has the same function as the view details control 523B.

[0177] When the first device continues to display the control interface of application C of the target device, that is, after displaying the interface shown in c in Figure 5, if the user wants to start / stop / switch playing the fitness course, the user can input an operation to the corresponding control in the application control area 533, thereby allowing the first device to control application C of the target device across devices to start / stop / switch playing the fitness course.

[0178] The user interface 54 shown in d of Figure 5 is an interface displayed after the first device receives an operation for switching to play a fitness course. The user interface 54 displays a device switching area 541, an application switching area 542 and an application control area 543.

[0179] Target device identifier 541A in device switching region 541 displays the same content as target device identifier 531A in device switching region 531, indicating that the target device remains a mobile phone. Controls 541B and 541C in device switching region 541 have the same functions as controls 531B and 531C in device switching region 531.

[0180] The identifiers of the applications in the application switching area 542 and the aforementioned application switching area 532 are the same.

[0181] Application control area 543 displays different content than application control area 533. Specifically, the target application's running status 543A in application control area 543 shows that the target application is providing a full-body fat burning class, while the target application in application control area 533 shows that the target application is providing a leg strength training class. The view details control 543B in application control area 543 has the same function as the view details control 533B.

[0182] When the first device continues to display the application C control interface of the target device, that is, after displaying the interface shown in Figure 5d, if the user wants to start / stop / switch playing the fitness course, the user can input an operation to the corresponding control in the application control area 543, thereby allowing the first device to control the application C of the target device across devices to start / stop / switch playing the fitness course.

[0183] Scenario 3: The user uses the tablet's camera app to take a video.

[0184] Specifically, when users use the camera application of a tablet to take videos, they usually need to make various gestures, but in the process of making gestures, the user cannot touch the tablet to take snapshots, start recording, end recording, etc. However, if the method described below in this application is adopted to control the tablet across devices by using the multi-device collaborative control function of the first device, it is convenient for users to control the tablet to take videos at will while making gestures. In this application, in addition to posing, users can also control the corresponding electronic device to take videos if they want to take photos or record videos during cooking, exercise, or speaking. Preferably, if the first device is a wearable device, the user can use the first device anytime and anywhere, and there will be no situation similar to approaching the first device to interrupt shooting and approaching the tablet.

[0185] FIG6 exemplarily shows a schematic diagram of an operation interface for using a multi-device collaboration function in a set of camera scenes.

[0186] In Figure 6, a is a user interface 61 when the first device controls application Y in the target device across devices. This user interface 61 may be the interface displayed after the first device enables the multi-device collaborative control function and sets the target device to a tablet and the target application to application Y. For the specific implementation method of enabling the multi-device collaborative control function and setting the target device and target application, please refer to the previous description of setting the mobile phone / tablet as the target device and setting application A / application X as the target application in Figures 3 and 4, and will not be repeated here.

[0187] The user interface 61 displays a device switching area 611 , an application switching area 612 , an application control area 613 , and the like.

[0188] The device switching area 611 displays the target device identifier 611A and controls 611B and 611C for switching the target device. Detailed description of the device switching area 611 can be found in the previous description of the device switching area 341, which will not be repeated here.

[0189] The application switching region 612 displays a group of application identifiers corresponding to the target device's applications, such as identifiers for Application X, Application Y, and Application Z. Application Y is a target application, which may be a default application of the first device or a user-selected application. For a detailed description of the application switching region 612, please refer to the previous description of the application switching region 342, and will not be elaborated on here.

[0190] The application control area 613 displays a control panel for the target application. The control panel includes, for example, an operating status 613A of the target application. The operating status 613A of the target application indicates that the target application is temporarily stopped and not running. When the target application is not running, the application control area 613 also displays a control 613B for controlling the target device to activate application Y.

[0191] When the first device displays a in Figure 6, the first device can control the running status of application Y in the target device across devices. For example, by operating the control 613B in the application control area 613 in a in Figure 6, the first device can control the target device across devices to start application Y.

[0192] The user interface 62 shown in b of Figure 6 is the interface displayed after the first device receives an operation for cross-device control of the target device to start application Y. The user interface 62 includes a device switching area 621, an application switching area 622, and an application control area 623.

[0193] The target device identifier 621A in device switching area 621 displays the same content as the target device identifier 611A in device switching area 611, indicating that the target device remains a tablet. Controls 621B and 621C in device switching area 621 have the same functions as controls 611B and 611C in device switching area 611.

[0194] The identifiers of the applications in the application switching area 622 and the aforementioned application switching area 612 are the same.

[0195] The application control area 623 displays different content than the aforementioned application control area 613. Specifically, the target application in application control area 623 is in the activated state, while the target application in application control area 613 is in the inactivated state. Furthermore, application control area 623 displays application Y's operating status information 623A and controls 623B for controlling the operating status of application Y. For example, operating status information 623A includes a preview of application Y on the target device, and controls 623B can be used to control application Y on the target device to take photos or record videos.

[0196] After the first device continues to display the application Y control interface of the target device, that is, after the first device displays b of Figure 6, if the user wants to start / stop taking photos or recording videos, he or she can input operations to the corresponding controls in the application control area 623, thereby enabling the first device to control the application Y of the target device across devices to start / stop taking photos or recording videos.

[0197] The user interface 63 shown in c of Figure 6 is the interface displayed after the first device receives an operation for recording (i.e., long pressing the control 623B for a preset duration, such as 1 second). The user interface 63 displays a device switching area 631, an application switching area 632, and an application control area 633.

[0198] The target device identifier 631A in device switching area 631 displays the same information as the target device identifier 621A in device switching area 621, indicating that the target device remains a tablet. Controls 631B and 631C in device switching area 631 have the same functions as controls 621B and 621C in device switching area 621.

[0199] The identifiers of the applications in the application switching area 632 and the aforementioned application switching area 622 are the same.

[0200] The application control area 633 displays different content from the application control area 623. Specifically, the running status information 633A of application Y in the application control area 633 shows that the target application is in the recording start state, while the target application in the application control area 623 is in the preview state.

[0201] After the first device continues to display the control interface of application Y of the target device, if the user wants to stop recording, he or she can input a click operation to the control 633B in the application control area 633, thereby allowing the first device to control application Y of the target device across devices to end recording.

[0202] The user interface 64 shown in d of Figure 6 is an interface displayed after the first device receives an operation for stopping recording. The user interface 64 displays a device switching area 641, an application switching area 642, and an application control area 643.

[0203] The target device identifier 641A in device switching area 641 displays the same information as the target device identifier 631A in device switching area 631, indicating that the target device remains a tablet. Controls 641B and 641C in device switching area 641 have the same functions as controls 631B and 631C in device switching area 631.

[0204] The application switching area 642 has the same identification as the application in the aforementioned application switching area 632 .

[0205] The application control area 643 displays different content from the aforementioned application control area 633. Specifically, the target application in the application control area 643 is in a recording completion state, while the target application in the application control area 633 is in a recording state.

[0206] When the first device continues to display the control interface of the target device's application Y, if the user wants to start / stop taking photos or recording videos, the user can input an operation to the control 643B in the application control area 643, thereby allowing the first device to control the target device's application Y across devices to start / stop taking photos or recording videos. The corresponding target application's running status 643A also displays the stopped recording status.

[0207] Scenario 4: The user uses the laptop's screen projection function to project the screen and give a speech.

[0208] Specifically, when a user uses a notebook to project a screen to a projection device to give a speech, he usually needs to be close to the notebook and then operate the corresponding controls of the notebook to control the document presented by the projection device to stop / continue playing, or play the next page, etc. However, if the method described below in this application is adopted to control the notebook across devices by adopting the multi-device collaborative control function of the first device, it is convenient for the user to control the notebook to turn pages at will when he is away from the notebook (such as interacting off-stage). Preferably, if the first device is a wearable device, the user can use the first device anytime and anywhere, and there will be no situation where he approaches the first device as he approaches the notebook.

[0209] Figure 7 exemplarily shows a schematic diagram of the operation interface using the multi-device collaboration function in a group of screen projection and lecture scenarios.

[0210] In Figure 7, a is a user interface 71 (control interface) of the first device displaying application M in the target device. This user interface 71 may be the interface displayed after the first device turns on the multi-device collaborative control function and sets the target device to a notebook and the target application to application M. Regarding the specific implementation method of turning on the multi-device collaborative control function and setting the target device and target application, please refer to the previous description of setting the mobile phone / tablet as the target device and setting application A / application X as the target application in Figures 3-4, which will not be repeated here.

[0211] The user interface 71 displays a device switching area 711 , an application switching area 712 , an application control area 713 , and the like.

[0212] The device switching area 711 displays the target device identifier 711A and controls 711B and 711C for switching the target device. Detailed description of the device switching area 711 can be found in the previous description of the device switching area 341, which will not be repeated here.

[0213] The application switching region 712 displays a group of application identifiers corresponding to the target device's applications, including, for example, identifiers for application L, application M, and application N. Application M is a target application, which may be a default setting on the first device or a user-selected application. For a detailed description of the application switching region 712, please refer to the previous description of the application switching region 342, and will not be elaborated on here.

[0214] The application control area 713 displays the target application's control panel, which includes, for example, the target application's operating status 713A and controls 713B. For example, the target application's operating status 713A may be presenting a first-quarter report, and the controls 713B include, but are not limited to, controls for presenting the next / previous page of a document and for ending playback. Furthermore, the target device will project the target application's presentation document onto the projection device for display.

[0215] When the first device displays a in Figure 7, the first device can control the running status of the application M in the target device across devices. For example, by operating the control 713B in the application control area 713 in a in Figure 7, the first device can control the target device across devices to play the next / previous page of documents or end playing the document, thereby causing the projection device end of the target device to also play the next / previous page of documents or end playing the document.

[0216] The user interface 72 shown in FIG7 b is the interface displayed after the first device receives an operation for controlling the target device to play the next page of the document across devices. The operation for playing the next page of the document can be clicking the control for playing the next page in the control 713B shown in FIG7 a, or can be an operation of a physical control on the first device (such as rotating a crown).

[0217] The user interface 72 displays a device switching area 721 , an application switching area 722 , an application control area 723 , and the like.

[0218] Target device identifier 721A in device switching area 721 displays the same information as target device identifier 711A in device switching area 711, indicating that the target device remains a laptop. Controls 721B and 721C in device switching area 721 have the same functions as controls 711B and 711C in device switching area 711.

[0219] The application switching area 722 has the same identification as the application in the aforementioned application switching area 712 .

[0220] The application control area 723 displays different content from the aforementioned application control area 713. Specifically, the target application's running status 723A, for example, indicates that the document being presented by the target application is a second-quarter report, while the target application's running status 713A in the application control area 713 indicates that the document being presented by the target application is a first-quarter report.

[0221] When the first device continues to display the control interface of application M of the target device, that is, after the first device displays b of Figure 7, if the user wants to continue switching to the next page / previous page, or end playback, it can be achieved by acting on the corresponding control in control 723B.

[0222] User interface 73 shown in FIG7 c is the interface displayed after the first device receives an operation for controlling the target device to play the next page of the document across devices. The operation for playing the next page of the document can be clicking the control for playing the next page in control 723B shown in FIG7 b, or can be an operation of a physical control on the first device (such as rotating a crown).

[0223] The user interface 73 displays a device switching area 731 , an application switching area 732 , and an application control area 733 .

[0224] Target device identifier 731A in device switching area 731 displays the same information as target device identifier 721A in device switching area 721, indicating that the target device remains a laptop. Controls 731B and 731C in device switching area 731 have the same functions as controls 721B and 721C in device switching area 721.

[0225] The application switching area 732 has the same identification as the application in the aforementioned application switching area 722 .

[0226] The application control area 733 displays different content from the application control area 723. Specifically, the target application's running status 733A in the application control area 733 may display a third-quarter report, while the target application in the application control area 723 displays a second-quarter report.

[0227] After the first device continues to display the control interface of application M of the target device, that is, after the first device displays c of Figure 7, if the user wants to continue switching to the next page / previous page, or end playback, it can be achieved by acting on the corresponding control in control 733B.

[0228] The user interface 74 shown in FIG7(d) is the interface displayed after the first device receives an operation for ending document playback. The operation for ending playback may be clicking the control for playing the next page in the control 733B shown in FIG7(c), or may be an operation performed on a physical control (e.g., a button) of the first device.

[0229] The user interface 74 displays a device switching area 741 , an application switching area 742 , and an application control area 743 .

[0230] Target device identifier 741A in device switching area 741 displays the same information as target device identifier 731A in device switching area 731, indicating that the target device remains a laptop. Controls 741B and 741C in device switching area 741 have the same functions as controls 731B and 731C in device switching area 731.

[0231] The application switching area 742 has the same identification as the application in the aforementioned application switching area 732 .

[0232] The application control area 743 displays different content from the aforementioned application control area 733. Specifically, the target application's running status 743A in the application control area 743 may display a document titled "Thank you for watching," while the target application's document titled "Third Quarter Report" in the application control area 733 may display a document titled "Third Quarter Report."

[0233] When the first device continues to display the control interface of application M of the target device, that is, after the first device displays d in Figure 7, if the user wants to continue switching to the next page / previous page, or replay, it can be achieved by acting on the corresponding control in control 743B.

[0234] Scenario 5: Users watch movies using video apps on smart screens.

[0235] Specifically, when users use video applications on smart screens to watch movies, they usually need to use a remote control to control the smart screen to switch movies, adjust the movie playback progress or playback volume, etc. However, if the method described below in this application is adopted to control the smart screen across devices by using the multi-device collaborative control function of the first device, it is convenient for users to quickly switch movies on the smart screen, adjust their playback progress or playback volume, etc., and avoid having to interrupt watching movies because of looking for the remote control. Preferably, if the first device is a wearable device, the user can use the first device anytime and anywhere, and there will be no need to search for the first device like searching for the remote control.

[0236] FIG8 exemplarily shows a schematic diagram of an operation interface for using a multi-device collaboration function when watching a movie.

[0237] In Figure 8, a is a user interface 81 (control interface) of the first device displaying application V in the target device. The user interface 81 may be the interface displayed after the first device turns on the multi-device collaborative control function and sets the target device to the smart screen and the target application to application V. Regarding the specific implementation method of turning on the multi-device collaborative control function and setting the target device and target application, please refer to the previous description of setting the mobile phone / tablet as the target device and setting application A / application X as the target application in Figures 3-4, which will not be repeated here.

[0238] The user interface 81 displays a device switching area 811 , an application switching area 812 , an application control area 813 , and the like.

[0239] The device switching area 811 displays the target device identifier 811A and controls 811B and 811C for switching the target device. Detailed description of the device switching area 811 can be found in the previous description of the device switching area 341, which will not be repeated here.

[0240] The application switching region 812 displays a group of application identifiers corresponding to the target device's applications, including, for example, identifiers for application U, application V, and application W. Application V is a target application, which may be a default application of the first device or a user-selected application. For a detailed description of the application switching region 812, please refer to the previous description of the application switching region 342, and will not be elaborated on here.

[0241] The application control area 813 displays a control panel of the target application, which includes, for example, the target application's operating status 813A and controls 813B. The target application's operating status 813A, for example, indicates that the movie "Movie B" is stopped. Controls 813B include, but are not limited to, controls for switching to the next or previous movie, controls for pausing or resuming movie playback, controls for adjusting playback progress, and controls for adjusting playback volume.

[0242] When the first device displays a in Figure 8, the first device can control the running status of the application V in the target device across devices. For example, by operating the corresponding controls in the control 813B, the first device can control the target device across devices to switch to the next / previous video, pause / resume playing the video, adjust the playback progress, or adjust the playback volume, etc.

[0243] The user interface 82 shown in FIG8(b) is displayed after the first device receives an operation for controlling the target device to play a video across devices. The operation for playing the video can be clicking the control for playing the video in the control 813B shown in FIG8(a), or can be an operation on a physical control (e.g., a button) on the first device.

[0244] The user interface 82 displays a device switching area 821 , an application switching area 822 , an application control area 823 , and the like.

[0245] The target device identifier 821A in the device switching area 821 displays the same content as the target device identifier 811A in the aforementioned device switching area 811, indicating that the target device remains the smart screen. The controls 821B and 821C in the device switching area 821 have the same functions as the controls 811B and 811C in the aforementioned device switching area 811.

[0246] The application switching area 822 has the same identification as the application in the aforementioned application switching area 812 .

[0247] The application control area 823 displays different content from the application control area 813. Specifically, the target application's running state 823A may be playing the movie "Movie B," while the target application's running state 813A may be stopped.

[0248] When the first device continues to display the control interface of application V of the target device, that is, after the first device displays b of Figure 8, if the user wants to continue to switch to the next / previous video, pause / resume playing the video, adjust the playback progress, or adjust the playback volume, etc., it can be achieved by acting on the corresponding controls in control 823B.

[0249] User interface 83 shown in FIG8 c is the interface displayed after the first device receives an operation for controlling the target device to play the next video across devices. The operation of switching to the next video can be performed by clicking the control for switching to the next video in control 823B shown in FIG8 b, or it can be performed by operating a physical control (such as a crown) on the first device.

[0250] The user interface 83 displays a device switching area 831 , an application switching area 832 and an application control area 833 .

[0251] The target device identifier 831A in the device switching area 831 displays the same content as the target device identifier 821A in the aforementioned device switching area 821, indicating that the target device remains the smart screen. The controls 831B and 831C in the device switching area 831 have the same functions as the controls 821B and 821C in the aforementioned device switching area 821.

[0252] The application switching area 832 has the same identification as the application in the aforementioned application switching area 822 .

[0253] The application control area 833 displays different content from the application control area 823. Specifically, the target application's running status 833A may be, for example, the state of playing "Movie B," while the target application's running status 823A may be the state of playing "Movie A."

[0254] When the first device continues to display the control interface of the target device's application V, that is, after the first device displays c in Figure 8 , if the user wants to continue switching to the next / previous video, pause / resume playing the video, adjust the playback progress, or adjust the playback volume, etc., it can be achieved through the corresponding controls in the operation control 833B.

[0255] The user interface 84 shown in FIG8(d) is displayed after the first device receives an operation for controlling the target device to stop playing a video across devices. The operation to stop playing the video can be performed by clicking the control for stopping playing the video in the control 833B shown in FIG8(c), or by operating a physical control (e.g., a button) on the first device.

[0256] The user interface 84 displays a device switching area 841 , an application switching area 842 , and an application control area 843 .

[0257] The target device identifier 841A in the device switching area 841 displays the same content as the target device identifier 831A in the aforementioned device switching area 831, indicating that the target device remains the smart screen. The controls 841B and 841C in the device switching area 841 have the same functions as the controls 831B and 831C in the aforementioned device switching area 831.

[0258] The application switching area 842 has the same identification as the application in the aforementioned application switching area 832 .

[0259] The application control area 843 displays different content from the aforementioned application control area 833. Specifically, the target application's running state 843A is, for example, a state where the movie "Movie A" is stopped, while the target application's running state 833A is, for example, a state where the movie "Movie A" is playing.

[0260] When the first device continues to display the control interface of the application V of the target device, that is, after the first device displays d in Figure 8, if the user wants to continue to switch to the next / previous video, pause / resume playing the video, adjust the playback progress, or adjust the playback volume, etc., it can be achieved through the corresponding controls in the operation control 843B.

[0261] The UI embodiments shown in the aforementioned Figures 3 to 8 only use several common application scenarios as examples to introduce the multi-device collaborative control method provided by this application. In different scenarios, the global control interface displayed by the first device may be different. For example, the display form of the application logo in the application switching area in the global control interface may be different, the display of the application status in the application control area may be different, the type of related controls may be different, etc., which may be adaptively changed according to the characteristics of the target application. This application does not give examples one by one here. Based on the UI embodiments shown in the aforementioned Figures 3 to 8, it can be known that after applying the multi-device collaborative control method, graphical interface and electronic device provided by this application, the user can quickly and conveniently switch the target device to be controlled, switch the target application to be controlled, and control the execution of tasks related to the target application through the global control interface in various scenarios. Especially in the scenario where multiple devices and multiple applications collaborate to perform tasks, thanks to the centralized control entrance, the user's operation can be simplified and the interactive experience can be improved.

[0262] It should be understood that each step in the above method embodiments provided herein can be implemented by hardware integrated logic circuits in a processor or by software instructions. The method steps disclosed in the embodiments of this application can be directly implemented as being executed by a hardware processor, or by a combination of hardware and software modules in a processor.

[0263] The present application also provides an electronic device, which may include: a memory and a processor, wherein the memory may be used to store a computer program; and the processor may be used to call the computer program in the memory so that the electronic device executes the method in any one of the above embodiments.

[0264] The present application also provides a chip system, which includes at least one processor for implementing the functions involved in the method executed by the electronic device in any of the above embodiments.

[0265] In one possible design, the chip system further includes a memory, which is used to store program instructions and data, and the memory is located inside or outside the processor.

[0266] The chip system can be composed of chips, or can include chips and other discrete devices.

[0267] Optionally, there may be one or more processors in the chip system. The processor may be implemented in hardware or software. When implemented in hardware, the processor may be a logic circuit, an integrated circuit, etc. When implemented in software, the processor may be a general-purpose processor implemented by reading software code stored in a memory.

[0268] Optionally, the memory in the chip system may be one or more. The memory may be integrated with the processor or may be provided separately from the processor, which is not limited in the embodiments of the present application. For example, the memory may be a non-transient processor, such as a read-only memory (ROM), which may be integrated with the processor on the same chip or provided on different chips. The embodiments of the present application do not specifically limit the type of memory or the configuration of the memory and the processor.

[0269] Exemplarily, the chip system may be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD) or other integrated chips.

[0270] The present application also provides a computer program product, which includes: a computer program (also referred to as code, or instruction), which, when executed, enables a computer to execute the method executed by the electronic device in any of the above embodiments.

[0271] The present application also provides a computer-readable storage medium storing a computer program (also referred to as code or instruction). When the computer program is executed, the computer executes the method executed by the electronic device in any of the above embodiments.

[0272] The various implementation modes of this application can be combined arbitrarily to achieve different technical effects.

[0273] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described herein are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive).

[0274] Those skilled in the art will appreciate that all or part of the process steps in the above-described method embodiments can be implemented by a computer program instructing the relevant hardware. The program can be stored in a computer-readable storage medium, and when executed, the program can include the process steps in the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0275] In short, the above description is only an embodiment of the technical solution of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made based on the disclosure of the present invention should be included in the scope of protection of the present invention.

Claims

1. A multi-device collaborative control method, characterized in that: The method is applied to a first device, and includes: Displaying a first interface, the first interface including a first area, a second area, and a third area; the first area including an identifier of a second device, the second area including identifiers of a first group of applications of the second device, and the third area including a control panel of a first application in the first group of applications; In response to the first operation, displaying the logo of the third device after the switch in the first area, displaying the logo of the second group of applications of the third device after the switch in the second area, and displaying the control panel of the second application in the second group of applications after the switch in the third area; In response to the second operation, displaying an identifier of the switched third application in the second area, and displaying a control panel of the switched third application in the third area; In response to the third operation, a first task corresponding to the third application is executed.

2. The multi-device collaborative control method according to claim 1, characterized in that: In response to the second operation, the method further includes: switching the identifier of the application in the selected state in the second area from the identifier of the second application to the identifier of the third application.

3. The multi-device collaborative control method according to claim 1 or 2, characterized in that: The first task includes any one or more of the following: Displaying the running details of the third application; Alternatively, the running state of the third application is controlled.

4. The multi-device collaborative control method according to claim 3, characterized in that: Controlling the running state of the third application includes any one or more of the following: controlling the running state of the third application in the third device, Alternatively, the running status of the third application in the first device is controlled.

5. The multi-device collaborative control method according to any one of claims 1 to 4, characterized in that: The third application is an audio and video application, and executing the first task corresponding to the third application includes: controlling the third device to pause, continue, or switch the playing audio and video.

6. The multi-device collaborative control method according to any one of claims 1 to 5, characterized in that: The third application is a shooting application, and the first task corresponding to the third application includes: controlling the third device to shoot images.

7. The multi-device collaborative control method according to any one of claims 1 to 6, characterized in that: The third application is an office application, and executing the first task corresponding to the third application includes: controlling the third device to pause, continue, or switch a presentation document.

8. The multi-device collaborative control method according to any one of claims 1 to 7, characterized in that: The third application is a social application, and executing the first task corresponding to the third application includes: answering / hanging up a call.

9. The multi-device collaborative control method according to any one of claims 1 to 8, characterized in that: Before the first operation, the method further includes: In response to a fourth operation, a first task corresponding to the first application is executed.

10. The multi-device collaborative control method according to any one of claims 1 to 9, characterized in that: The control panel includes: application information and controls for controlling applications.

11. The multi-device collaborative control method according to any one of claims 1 to 10, characterized in that: The first operation includes any one or more of the following: an operation acting on the first area; Alternatively, when the first device is a watch, the operation acts on the crown of the watch.

12. The multi-device collaborative control method according to any one of claims 1 to 11, characterized in that: The second operation includes any one or more of the following: an operation acting on the second area; Alternatively, when the first device is a watch, the operation acts on the crown of the watch.

13. The multi-device collaborative control method according to any one of claims 1 to 12, characterized in that: The third operation includes any one or more of the following: an operation acting on the third area; Alternatively, when the first device is a watch, the operation acts on the crown of the watch.

14. The multi-device collaborative control method according to any one of claims 1 to 13, characterized in that: The area of the first region is smaller than that of the second region, and the area of the second region is smaller than that of the third region.

15. The multi-device collaborative control method according to any one of claims 1 to 14, characterized in that: The first area is located on the upper side of the first device screen, the second area is located on the left side of the first device screen, and the third area is located on the right side of the first device screen.

16. The multi-device collaborative control method according to any one of claims 1 to 15, characterized in that: The first device is a wearable device.

17. The multi-device collaborative control method according to any one of claims 1 to 16, characterized in that: The method further includes: before displaying the first interface, the first device establishing a connection with the second device.

18. The multi-device collaborative control method according to claim 17, characterized in that: The establishing of a connection between the first device and the second device specifically includes: Displaying a second interface, the second interface including an identifier of a global control function and an identifier of the second device; In response to an operation of dragging the global control function identifier to the identifier of the second device and releasing it, the first device establishes a connection with the second device.

19. The multi-device collaborative control method according to any one of claims 1 to 18, characterized in that: The method further includes: before or after displaying the identifier of the third device, the first device establishing a connection with the third device.

20. An electronic device, characterized in that: The electronic device includes one or more memories and one or more processors; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the method described in any one of claims 1-19.

21. A communication system, characterized in that: The communication system includes a first device, a second device and a third device, the first device includes one or more memories and one or more processors; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to cause the first device to execute the method according to any one of claims 1 to 19.

22. A chip, applied to electronic equipment, characterized in that: The chip includes one or more processors, and the processors are configured to call computer instructions so that the electronic device executes the method according to any one of claims 1 to 19.

23. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on an electronic device, the electronic device is caused to execute the method according to any one of claims 1 to 19.

Images